CONTRAST COMPENSATION FOR BACK-LIT AND FRONT-LIT COLOR FACE IMAGES VIA FUZZY LOGIC CLASSIFICATION AND IMAGE ILLUMINATION ANALYSIS

Transcription

1 CONTRAST COMPENSATION FOR BACK-LIT AND FRONT-LIT COLOR FACE IMAGES VIA FUZZY LOGIC CLASSIFICATION AND IMAGE ILLUMINATION ANALYSIS CHUN-MING TSAI 1, ZONG-MU YEH 2, YUAN-FANG WANG 3 1 Department of Computer Science, Taipei Municipal University of Education, Taipei 100, Taiwan, ROC 2 Department of Mechatronic Technoloy, National Taiwan Normal University, Taipei 106, Taiwan, ROC 3 Department of Computer Science, University of California, Santa Barbara, CA 93106, USA cmtsai@tmue.ed.tw, zonmu@ntnu.edu.tw, yfwan@cs.ucsb.edu Abstract: Conventional contrast enhancement methods have two shortcomins. First, most of them do not produce satisfactory enhancement results for face imaes with back-lit or front-lit. Second, most of them need transformation functions and parameters which are specified manually. Thus, this paper proposes an automatic and parameter-free contrast compensation alorithm for color face imaes. This method includes: RGB color space is transformed to YIQ color space. Fuzzy loic is used to classify the color imaes into back-lit, normal-lit, and front-lit cateories. Imae illumination analysis is used to analyze the imae distribution. The input imae is compensated by piecewise linear based compensation method. Finally, the compensation imae is transformed back to RGB color space. This novel compensation method is automatic and parameter-free. Our experiments included back-lit and front-lit imaes. Experiment results show that the performance of the proposed method is better than other available methods in visual perception measurements. Keywords: Contrast compensation; Fuzzy loic classification; Imae illumination analysis; Parameter-free; Color face imaes 1. Introduction Contrast enhancement is developed to adjust the quality of an imae for better human visual perception [1]. It is a very important preprocessin step for tasks in imae processin, video processin, medical imae processin, aerial imae processin, and computer vision. Current contrast enhancement alorithms based on spatial domain techniques can be divided into four main types: lobal, local, hybrid, and fuzzy methods. Global methods enhance the imae from the information of an entire imae. Duan and Qiu [2] divided the luminance rane [0, 255] into 256 intervals usin a hierarchical division procedure and used a control parameter to control the mappin. Sun et al. [3] proposed a dynamic specific historam alorithm to do contrast enhancement for a real-time system due to its simplicity. Local methods enhanced the imae for each pixel accordin to the information of its own and its neihbor. Chatterji and Murthy [4] proposed an adaptive contrast enhancement for color imaes. However, in their method, two parameters (e.., enhancement function and reion size) are needed to determine. Meylan and Süsstrunk [5] used a Retinex-based adaptive filter to enhance natural color imaes, and their results showed that the color imae with halo area could be enhanced. However, it needed to choose the appropriate filter size to reduce halos imaes and to introduce lobal tone mappin for extremely hih dynamic rane imaes. Hybrid methods combined both lobal and local approaches. In these methods, an imae is divided into non-overlay or overly reions and each reion is conquered by lobal methods. Kim et al. [6] used partially overlapped sub-block HE (POSHE) technique to achieve contrast enhancement. However, the visual quality and computational speed-up is a trade-off problem. Furthermore, the sub-block divisor and the overlap step divisor are determined in advance. Recently, Menotti et al. [7] proposed a Multi-HE technique, which consists of decomposin the input imae into several sub-imaes, and applyin the classical HE process to each one. Their method can preserve more the brihtness and produce more natural lookin imaes than the other HE methods. However, their method need to set the weihtin constant of the cost function in advance. Fuzzy contrast enhancement [8] is based in the concept of mappin a rayscale imae onto a fuzzy plane. This method uses some form of membership function to enhance imae. The membership function characterizes some property of an imae (e.. ediness). The process is known as imae fuzzification. The membership values are then modified in some manner to improve the contrast. The modified /08/$ IEEE 3563

2 membership values are then inversely transformed throuh the process of defuzzification to produce the enhanced imae. Fuzzy enhancement alorithms are certainly suited to certain applications. But it needs to determine what these are in advance. Furthermore, the fuzzy enhancement methods often need adjustable parameters (the choice of membership function) which may affect the quality of the result. Historam Equalization (HE) and its variations are one of the most commonly used alorithms to perform contrast enhancement due to its simplicity and effectiveness [1], [3], [7], [9], [10], [11]. Historam equalization can be extended to process color imaes [11], [12]. However, it has many shortcomins: (1) it cannot preserve the brihtness illumination. (2) The result of historam equalization produced annoyin side effects, such as overhead brihtness enhancement, white noise, and some unnatural results [3]. (3) It chaned the hue. (4) HE and its variation methods produce unnatural lookin imaes [7]. However, if imaes with back-lit or front-lit, the above described methods cannot both produce satisfactory enhancement results. Herein, we will propose a contrast compensation method to enhance the imaes with back-lit and reduce the imaes with front-lit. The back-lit imaes are defined as the imaes with briht backround and dark foreround. In particular, the foreround is located at dark illumination. The front-lit imaes are defined as the imaes with dark backround and briht foreround. In particular, the foreround is located at briht illumination. The conventional enhancement methods determine the enhancement transformation function and parameters by human beins in advance. Furthermore, these methods only enhance the imae with back-lit and can not reduce the imae with front-lit. Therefore, in this paper a contrast compensation method via fuzzy loic classification to classify the imae and imae illumination analysis is proposed to determine the transformation function automatically. In particular, the parameters of the transformation function are chosen automatically, too. In other words, it does not require the intervention by a human operator. Moreover, the proposal method can enhance the imaes with back-lit and reduce the imaes with front-lit. 2. Fuzzy loic classification When takin a picture, the photoraphic object is placed at the center of imae [13]. Thus, the spatial characteristics of an imae are used to determine the input imae is back-lit or front-lit. The imae can be semented into two areas: boundary backround area and center foreround area by spatial position sementation method (c) (Fi. 1). The luminance rane of the boundary backround area (BB) and the center foreround area (CF) are used to determine the input imae is back-lit or front-lit. If the luminance rane of the boundary backround area is larer than the center foreround area, the input imae is back-lit imae. Otherwise, the input imae is front-lit imae. Due to the various illumination property and content of imaes, these two variables (BB and CF) have different derees of reliability. Thus, the fuzzy inference method is used for these two variables to determine the input imae is which class (IC, back-lit or front-lit). The luminance rane of boundary backround area (BB) and center foreround area (CF) are defined as follows: if (ut f < 128) BB = F b (ut b + k*std b ); CF = F f (ut f k*std f ) else BB=F b (ut b - k*std b ); CF = F f (ut f + k*std f ) (1) where ut f, std f, ut b, and std b is the averae luminance of the center foreround area, the standard deviation luminance of the center foreround area, the averae luminance of the boundary backround area, and the standard deviation luminance of the boundary backround area, respectively. F b and F f are membership function which converts BB and CF into a fuzzy deree, respectively. The constant k is used to consider the variance of the boundary backround area and the center foreround area. Here, k is set as 0.5. The middle luminance value in ray level is 128. Thus, this value is used to determine the center foreround area is prone to dark or briht ray level. Fuzzy inference systems [14] have been successfully applied in fields such as automatic control, data classification, decision analysis, expert systems, and computer vision. Fuzzy inference is the process of formulatin the mappin from a iven input to an output usin fuzzy loic. The fuzzy inference rules are characterized by a collection of fuzzy membership Low Medium Fiure 1. Fuzzy loic classification: the division of an input imae and, (c), and (d) are membership functions of the fuzzy sets of BB, CF, and IC, respectively. (d) Hih BB 3564

3 functions, loical operations, and IF-THEN rules. The antecedents and consequents involve linuistic variables. In the proposed fuzzy inference scheme for imae classification, there are two input variables, BB and CF, and one output variable IC. The two input variables have three fuzzy sets, L (low illumination), M (medium illumination), and H (hih illumination). The output variable has three fuzzy sets, BL (back-lit), FL (front-lit), and NL (normal-lit). The membership functions are used for the fuzzy sets of BB, CF, and IC are shown in Fis. 1-(d). Therefore, nine fuzzy interference rules are used here. Two representative rules are enumerated as follows: 1. If BB is L and CF is H then IC is FL. 2. If BB is H and CF is L then IC is BL. Based on the center of area (COA) defuzzification method [15], the value of the IC correspondin to two iven BB and CF values are obtained from the fuzzy inference. The inferred IC value represents the final estimated classification deree of the input imae. That is, to determine whether the input imae is back-lit imae or front-lit imae. 3. Imae illumination analysis Each color imae can be represented by Gaussian mixtures a more capable model to describe imae illumination distributions. To reduce the computation time, the imae illumination distribution at luminance domain is analyzed. Herein, each sinle Gaussian distribution is expressed as one peak and two valleys at luminance component. The luminance of a color imae is computed first. Then, the luminance historam H Y (x k ) is computed. The historam of an imae with brihtness levels in the rane [0, 255] is a discrete function H Y (x k ) = n k, where x k is the kth brihtness level and n k is the number of pixels in the imae havin brihtness level x k. Finally, Gaussian smoothin filter is applied to smooth the oriinal historam to obtain the reliable peaks and valleys. Thus the unreliable peaks and valleys are removed. The method of the Gaussian smoothin filter is described as follows. The Gaussian convolution of a luminance historam H Y (x) depends upon both x and, namely, the Gaussian standard deviation. The convolution function S HY (x, ) is provided by Eq. (2), S HY ( x, ) HY ( x) ( x, ) = HY ( u) ( x u, ) 2 ( x u) = HY ( u) e du 2π = du where * denotes the convolution operator and (x-u, ) is the Gaussian function. The deree of smoothin is (2) Fiure. 2. An example for imae illumination analysis to extract the luminance distributions. Oriinal historam. Selection of imae luminance distribution. controlled by the standard deviation of the Gaussian function. The larer the standard deviation, the smoother the function S HY (x, ) is. The standard deviation is referred to [16]. They decided the standard deviation automatically. Equation (2) is employed to convolute the historam H Y (x) that provides smoothin historam. The objective of choosin the standard deviation is to smooth the most frequent ripples of the historam and to leave the sinificant modes. That is, the major peaks in the oriinal historam can be distinuished. The number of peaks in the smoothed historam is considered as the number of peaks in the oriinal historam. The selection of imae luminance distribution is described as follows. Here one condition is assumed that luminance distributions in imaes are multi-gaussian. After the small peaks and valleys have been removed, the averae differences are employed as the first derivation with which to determine the major peaks and valleys. The averae difference in point x is defined by 1 ' 1 SHY ( x + i) SHY ( x i) (3) SHY ( x) = 1 i= 1 2 i A peak is defined as a positive to neative crossover in the first derivation of the smoothed historam. Furthermore, a valley is defined as a neative to positive crossover. All peaks and valleys from the first derivation of the smoothed historam are discovered. In cases where the peaks and valleys are too close, they will be removed if the distance between a valley and a peak is less than the standard deviation. The remainin peaks are the candidates of the luminance distribution in the imae. Fiure 2 shows an example for imae illumination analysis. The input imae is processed by transformin RGB into YIQ, ettin the luminance historam, smoothin the historam, and selectin the color distributions. Each color distribution is selected by one peak and two valleys in luminance domain. The oriinal historam is shown in Fi. 2. The smoothed historam and the selection of the luminance distribution are depicted at Fi. 2. There are 3565

4 Fiure 3. Piecewise linear transformation function (k = 3, k is the number of the input parameters). five luminance distributions. The rane of each luminance distribution is represented by one peak and its two valleys. 4. Automatic and parameter-free compensation The proposed compensation alorithm is based on piecewise linear transformation [1]. The axiom of piece-wise linear transformation and how to build transformation function and parameters automatically are described as follows. The piecewise linear transformation (PLT) is characterized by 2k parameters for k-1 line sements. When the parameters are iven, the transformation line sements will be determined. That is, if iven the startin position of input luminance {x k, k =0, 1... k} and the startin position of output luminance {y k, k=0, 1... k}, the k-1 transform functions T k-1 (x) will be: ( yk yk 1) T k 1 ( x) = ( x xk 1) + y (4) k 1 ( xk xk 1) For example, if k = 3, form the T 0 (x), T 1 (x), and T 2 (x) transformation functions are shown in Fi. 3. Four input parameters and four output parameters will be specified first manually. To determine these parameters and line sements are critical for the satisfactory result of the contrast compensation. In conventional applications, parameters and line sements are manually chosen case by case. To solve this problem, an automatic and parameter-free contrast compensation alorithm is proposed in the followin. The conventional PLT-based imae enhancement needs to set the number of the line sement and the values of the input and the output parameters in advance. How to decide the number of line sements automatically? Here, the number of the imae luminance distribution is used to represent the number of the line sements. If the number of the imae luminance distribution is k, then the number of the line sement is k+1. How to decide the input luminance parameter automatically? Here, the location of the valley is used to represent the input luminance parameter. The different imae may have different distribution of valleys. For example, there are six line sements in the Fi. 3. The input luminance parameters are 0, 11, 51, 93, 126, 154, and 166. The location of these valleys and the parameters depend on which imae is employed. How to determine the output luminance parameters automatically? The output parameters depend on which imae is used. The determinin method of input and output parameters are described in the followin. After the luminance distributions have been analyzed, many peaks are produced. If the color imae has k luminance distributions {p 1, p 2 p k } (peak is represented by p,) then the number of the line sement is k+1. Each distribution is bounded by two valleys indicated as v. That is, the k luminance distributions have {v 0, v 1 v k } valleys. These valleys are used to be the input parameters {x 0, x 1 x k } of the piecewise linear transformation function. The output parameters {y 0, y 1 y k } are defined as follows: v k y = Pr( x) 255 (5) k x= s where Pr(x) = n x /n is the probability of the xth luminance (ray level), n is the total number of pixels in the imae and n x is the number of times this x level appears in the imae. The parameter s is the start luminance of the input imae. The equation (5) is used to enhance the imae with normal-lit. In order to enhance back-lit or front-lit imaes, Eq. (5) must be adjusted. For the back-lit imae, the foreround is located at dark illumination. The illumination value of black is smaller than 60. This set value 60 is accordin to the property of human vision perception statin that the brihtness under 60 ray-level will be rearded as darkness [16]. Thus, Eq. (5) is adopted to shift the darkest illumination to briht illumination. That is, the dark illumination can see more clearly. Herein, we introduce a BlackShift parameter to resolve this problem. The BlackShift parameter sets as BlackShift = 60 - MaxDarkPeak. The MaxDarkPeak is the max peak which locates at the dark illumination. Thus, the output parameters {y 0, y 1 y k } of the back-lit imaes are defined as follows: v k y BlackShift + Pr( x) 255 (6) k = x= s For the front-lit imae, the foreround is located at briht illumination. The backround is located at dark illumination. In particular, the foreround is lihted by the sunliht. That is, the foreround is too briht to see. The briht luminance distributions need to be transformed from briht illumination spread to dark illumination. Thus, we introduce a WhiteShift parameter to resolve this problem. The WhiteShift parameter sets as WhiteShift = MaxBrihtPeak This set value 223 is accordin to the property of human vision perception statin that the brihtness above 223 ray-level will be rearded as 3566

5 brihtness. In particular, the skin of the face is influenced by the illumination. If the illumination is brihter, the color of the skin will be near to briht. Thus, the WhiteShift parameter is used to compress the brihtness. The transformation function is desined as similar to Eq. (6). Thus, the output parameters {y 0, y 1 y k } of the front-lit imaes are defined as follows: v k yk = Pr ( x) 255 WhiteShift (7) x= s 5. Experimental results and discussion The proposed contrast compensation alorithm for color face imaes was implemented as a Windows-based application on a Pentium (R) D 3.00GHZ and 504MB PC. The experiments use various color face imaes. Most of the imaes are obtained from Oulu Face Video Database [17], internet, and capture by ourselves. Some examples are presented as follows. The compensation results for back-lit imae are shown in Fi. 4. A back-lit imae is shown in Fi. 4 and its historam is shown in Fi. 4. The result by our method is shown in Fi. 4(c). Fiure 4(d) is the historam of Fi. 4(c). The result by historam equalization is shown in Fi. 4(e). Fiure 4(f) is the historam of Fi. 4(e). The result by contrast stretchin is shown in Fi. 4(). Fiure 4(h) is the historam of Fi. 4(). From Fi. 4, the result by contrast stretchin is similar to oriinal imae. Thus, the contrast stretchin cannot enhance the back-lit imae. The result by historam equalization is too briht. From visual perception, the result of the proposal method is better than the result of the historam equalization. The compensation results for front-lit imae are shown in Fi. 5. A back-lit imae is shown in Fi. 5 and its historam is shown in Fi. 5. The result by our method is shown in Fi. 5(c). Fiure 5(d) is the historam of Fi. 5(c). This historam is compared with the oriinal historam. The proposal method reduces the briht illumination. The result by historam equalization is shown in Fi. 5(e). Fiure 5(f) is the historam of Fi. 5(e). The result by contrast stretchin is shown in Fi. 5(). Fiure 5(h) is the historam of Fi. 5(). From Fi. 5, the result by contrast stretchin is similar to oriinal imae. Thus, the contrast stretchin cannot enhance the front-lit imae. The skin enhancement result by historam equalization is still too briht. The color of the polo shirt by our method is superior to historam equalization does. From visual perception, the foreround result of the proposal method is better than historam equalization does. (c) (e) () Fiure. 4. Compensation results for back-lit imae. a back-lit imae. The historam of. (c) Result by our method. (d) The historam of (c). (e) Result by historam equalization. (f) The historam of (e). () Result by contrast stretchin. (h) The historam of (). 6. Conclusions This study has presented a contrast compensation method by fuzzy loic classification and imae illumination analysis for color face imaes. The input imaes are classified by fuzzy loic classification method into back-lit, normal-lit, and front-lit first. Then, the illumination of the input imae was analyzed to obtain the imae illumination distributions. Each distribution is represented by one peak and two valleys. These peaks and valleys are used to be the parameters of the piecewise linear transformation. The performance analysis indicated that our method is efficient and effective by comparin with historam equalization and contrast stretchin. (d) (f) (h) 3567

6 Acknowledements This work was supported by the Ministry of Economic, R.O.C., under Grants MOEA-96-EC-17-A-02-S1-032 and National Science Council, R.O.C., under Grants NSC E References (c) (e) () Fiure. 5. Compensation results for front-lit imae. a front-lit imae. The historam of. (c) Result by our method. (d) The historam of (c). (e) Result by historam equalization. (f) The historam of (e). () Result by contrast stretchin. (h) The historam of (). [1] Gonzalez R. C. and Woods R. E., Diital Imae Processin, 2nd Edition, Prentice Hall, (d) (f) (h) [2] Duan J. and Qiu G., Novel historam processin for colour imae enhancement, Proc. of ICIM2004 Conf., pp , [3] Sun C. C., Ruan S. J., Shie M. C., and Pai T. W., Dynamic contrast enhancement based on historam specification, IEEE Trans. on Cons. Elec., Vol. 51, No. 4, pp , [4] Chatterji B. N. and R Murthy. N., Adaptive contrast enhancement for color imaes, Proc. of ICICS1997 Conf., Vol. 3, pp , [5] Meylan L. and Süsstrunk S., Color imae enhancement usin a retinex-based adaptive filter, Proc. of CGIV2004 Conf., Vol. 2, pp , [6] Kim J. Y., Kim L. S., and Hwan S. H., An advanced contrast enhancement usin partially over-lapped sub-block historam equalization, IEEE Trans. on CSVT, Vol. 11, No. 4, pp , [7] Menotti D., Najman L., Facon J., and de A. Araujo A., Multi-historam equalization methods for contrast enhancement and brihtness preservin, IEEE Trans. Cons. Elec., Vol. 53, No. 3, pp , [8] Chen H. D. and Xu Huijuan, A novel fuzzy loic approach to contrast enhancement, Pattern Reconition, Vol. 33, No. 5, pp , [9] Chen S. D. and Ramli A. R., Contrast enhancement usin recursive mean-separate historam equalization for scalable brihtness preservation, IEEE Trans. Cons. Elec., Vol. 49, No. 4, pp , [10] Chen S. D. and Ramli A. R., Minimum mean brihtness error bihistoram equalization in contrast enhancement, IEEE Trans. Cons. Elec., Vol. 49, No.4, pp , [11] Bockstein I. M., Color equalization method and its application to color imae processin, J. Opt. Soc. Am., Vol. 3, No. 5, pp , [12] Pichon E., Niethammer M., and Sapiro G., Color historam equalization throuh mesh deformation, Proc. of ICIP 2003 Conf., Vol. 2, pp , [13] Chin C. L. and Lin C. T., Detection and compensation alorithm for backliht imaes with fuzzy loic and adaptive compensation curve, Inter. Journal of Pattern Reconition and Artificial Intellience, Vo. 19, No. 8, pp , [14] Jan J.-S. R. and Sun C.-T., Neuro-Fuzzy and Soft Computin: A Computational Approach to Learnin and Machine Intellience, Prentice Hall, [15] Yeh Zon-Mu and Li Kuei-Hsian, A Systematic Approach for Desin of Multistae Fuzzy Control Systems, International Journal of Fuzzy Sets and Systems. Vol. 143, pp , [16] Tsai C. M. and Lee H. J., Binarization of color document imaes via luminance and saturation color features, IEEE Trans. IP, Vol. 11, No. 4, pp , [17] Martinkauppi B., Soriano M., Huovinen S., and Laaksonen M., Face video database, Proc. of CGIV2002 Conf., Poitiers, pp , April