Face Hallucination Based on Eigentransformation Learning

Transcription

1 Advanced Science and Technology etters, pp Face allucination Based on Eigentransformation earning Guohua Zou School of software, East China University of Technology, Nanchang, , China Abstract. n this paper, we study face hallucination which refers to inferring a high-resolution (R) face image from the input low-resolution (R) one. We advance an eigentransformation method [1] based on principal component analysis (PCA) for face hallucination by exploring the local geometry structure of data manifold and learning a specified eigentransformation model for each observation image. Firstly, we select some samples that are most similar to the input image patch. Secondly, a local patch-based eigentransformation method that can capture the local geometry structure is introduced for modeling the relationship between the R and R training samples. What s more, we also take a post-processing approach to refine the hallucinated results. Experimental results show that the proposed algorithm generates hallucinated face images with good quality and adaptability. Keywords: face hallucination, resolution enhancement, eigentransformation, noise face image 1 ntroduction With the deepening construction of Safe City, surveillance cameras which provide a powerful support for both finding criminal suspects and tracing their routes are already prevalent in our daily life. owever, under the circumstances of long distance between the camera and the object or bad weather, or the light change etc., the resolution of interested faces captured by the surveillance cameras is always low, which becomes a primary obstacle to face identification and recognition. Therefore, in order to gain detailed facial features for recognition, it is necessary to do resolution enhancement which use the technology called face hallucination. The goal of face hallucination is to generate high-resolution images with fidelity from low-resolution ones. Finding a high resolution image, given a low-resolution input, is an underconstrained problem: many images can put out the input after being smoothed and down-sampled. t is natural to divide super resolution work into two categories, which the constraints come from a direct or indirect correspondence. Early work on videobased super resolution includes [2, 3]. n 2000 Face hallucination was first proposed by Baker et al. [4]. Gradient distribution was learned at first time. They then employed Bayesian theory to infer the R face image. After that, a number of face SSN: AST Copyright 2016 SERSC

2 Advanced Science and Technology etters hallucination techniques have been proposed in recently years [5]. Wang and Tang [1] developed a face hallucination algorithm through eigentransformation. Principal component analysis (PCA) [4] was used to fit the input face image as a linear combination of the R training face images n this paper we propose to use the local geometry structure of the input R patch to guide the eigentransformation learning. What s more, we also take a postprocessing approach to refine the hallucinated results. 2 Basic Definitions of Face allucination Method et denote the high-resolution and and stands for low-resolution face images respectively. We develop a mixture model combining a global parametric model called the global face image which carries the common features of face, g l and a local nonparametric one called the local feature image which records the local individualities. The full-resolution face image is their sum (1) g l f is reduced from by a factor of s, we compute by, s 1 s 1 1 ( m, n) ( sm i, sn i) s 2 i 0 j 0 where s is always an integer. We take s = 6 unless otherwise specified. Eq. (2) combines a smoothing step and a down-sampling step, more consistent with image formation as integration over the pixel. To simplify the notation, if and are N-D and M-D long vectors. Eq. (2) can be simplified as (2) A (3) Where A is a M*N matrix, each row vectors in A smooths a s*s block in one pixel in. g Since is the low-frequency part of, the global face main part of A and the local features Mathematically, we have A g l to contributes the lie on the high-frequency band. A (4) A 0 (5) l Copyright 2016 SERSC 33

3 Advanced Science and Technology etters To compute from full of uncertainty. t is clear that many is straightforward in Eq. (3), but the inverse process is satisfy the constraint of Eq. (3). Thus we should find the optimal one to maximize the posterior probability. So the problem becomes to p( ) p( ) p( ) p ( ) uman face is one kind of object in highly structured, and they are very similar with each other. For the patches at a position on the entire facial images, we assume that they have the same position-patch structures and linear mapping. The details of our proposed method will be described in the following sections. (6) 3 Face allucination Based on Eigentransformation earning PCA is one kind of linear analysis method. n most cases when PCA is used, the random variable is regarded as a composition of two parts: the principal components and an unmodeled residue which is always assumed independent of the former. owever the face image we observe is in a high-dimensional nonlinear space, even if we put it into small pieces according to position. The space composed of face images or face patches isn t our usual said of European space, but should be a nonlinear and smooth manifold, which has an important character of local linearity. That is to say, for local manifold in a very small subset, we can assume that it is local European. To capture the local geometry of the data manifold, we plan to choose the K nearest neighbors in all patches with the same position according to the European distance. That is as below, where m is the number of training images. 2 d,1 i l i i m (7) 2 The K nearest neighbors can be represented by two matrixes and k k, where K is the number of neighbors, k is the index of the K nearest neighbors of input R patch in R training set and the R neighbor patches share the same index with R neighbor patches. According to the linear mapping in the eigentransformation algorithm, ˆ l c m (8) l k l ˆ h c m (9) h k h Where ˆl is the reconstruction of an R input with respect to the R training samples and ˆh is the corresponding reconstructed R image, 34 Copyright 2016 SERSC

4 Advanced Science and Technology etters ˆ T m E E ( ˆ m ) l l l l l l c K (10) c V V ( ) ( m ) (11) 1 T T l l k i l Using the same combination coefficients as demonstrated in Eq.8-9, the R face image can be reconstructed. The quality of reconstructed images is much better than that of R images. owever, all the theoretical bases mentioned above do not demand exact equality between the R patch and its reconstructed version. With further effect of noise added, there are still deviations between the original R image and our reconstructed result. We can eliminate these deviations by appending a correction step as Yang did in [7]. 4 Experiment Results n this experiment, we use the database contains 200 images from 100 individuals (50 men and 50 women). Among them, 80 individuals are randomly chosen as the training set, and the remaining 20 are as testing set. All the R facial images are aligned by the two eyes and fixed at pixels. R images are generated by blurring of averaging filter and down-sampling with a factor of 4. n our method, we set the size of R image patch to pixels with 8 pixels overlapped, and the variance accumulation contribution rate of PCA is set to n the preprocessing, the number of iterations suggests 10, reflaxation factor can be 0.05, and η is 0.5. Note that all parameters are set empirically. All the results will be evaluated by the two common used criterion PSNR and SSM, which is good at capturing the loss of image structure between reconstructed image and original R image. n order to assess the influence of nearest neighbor number K on the result of the proposed method, we conduct experiments on different K with different noise levels and show performance in terms of PSNR and SSM from 10 to 400. As can be seen from the result, the nearest neighbor number K plays an important role in the proposed method. By selecting a relatively small value of K, e.g., K = 75, the proposed method achieves the best performance. When the value of K exceeds 75, the performances decrease sharply. When K=360, it reduces to patch eigentransformation learning approach without sample selection. Experiments above demonstrate the efficiency of the proposed local patch based eigentransformation learning with K nearest neighbors searching. Table 1. PSNR(dB) and SSM results of average of all 40 test images. Methods Criterions Gaussian noise a=5 a=10 Wang[1] PSNR SSM NE PSNR SSM Copyright 2016 SERSC 35

5 Advanced Science and Technology etters SR PSNR SSM CR PSNR SSM ours PSNR SSM improvements PSNR SSM We compare the proposed approach with bicubic interpolation method and five other state-of-art learning-based face hallucination algorithms, i.e. Wang et al. [1], NE [6], SR, cr [9], and our proposed DE method. Note that in our experiments we carefully tune the parameters of these comparison methods to get the best performance. Table 1 tabulates the objective results in terms of average PSNR and SSM of all 40 test face images. We can observe that our method performs much better than all other methods when images are contaminated with noise. The average improvements over the second best approaches are 1.80 db and 3.48 db. We can also see the subjective reconstruction results with different noise level in Fig.1. We can observe that bicubic interpolation method produces serious blurry faces with noise preserved. The results of Wang s method are showed with obvious extra outliners and SR show obvious dark spots and some faces even still keep lots of noise, just like bicubic interpolation method. The results of some methods, like cr [9], appear to lack of details and edge blurring especially for mouth and eyes, leading to dissimilarity to original R image. These phenomenon become more serious after noise level rises to σ=10. At the same time, the details in areas of mouth, nose and eyes reconstructed by our method can be more similar to R image than other approaches. Both the objective and subjective results show that our proposed method can be applied to hallucinate the details of facial features from a R face image especially with strong noise. Fig. 1. Results of our algorithm compared to other methods. From left to right columns: (a) input R image; (b) Bicubic; (c) Wang et al. [1]; (d) NE [6]; (e) SR; (f) cr [9]; (g) our method; (h)original R image. 36 Copyright 2016 SERSC

6 Advanced Science and Technology etters 5 Conclusions n this paper, we propose a new method for face hallucination via f eigentransformation learning. Considering the local linearity of manifold, we pick out some samples that are most similar to the input image patch and then use the tailored training set to perform the face hallucination reconstruction process. After reconstruction, we propose a processing to optimize the reconstructed face and to guarantee the consistency between down-sampled reconstructed R face image and input R face image. Although we only use a small database and a simply face alignment algorithm, the results already reveal the potential of our algorithm for hallucinating faces and we leave that to our future work. Acknowledgements, The paper is supported by the Science & Technology Support Program of Nanchang City of China under Grant No. 2014ZZC010. References 1. Wang X., Tang, X.: allucinating face by eigentransformation, EEE Trans. Systems, Man, and Cybernetics. Part C, vol. 35, no. 3, pp , uang, T. S., Tsai, R. Y.: Multi-frame image restoration and registration. Advances in Computer Vision and mage Processing, 1: , Martinez, D.: Model-based motion estimation and its application to restoration and interpolation of motion pictures. PhD thesis, Massachusetts nstitute of Technology, Copyright 2016 SERSC 37