Comparing Image Representations for Training a Convolutional Neural Network to Classify Gender

Transcription

1 2013 Frst Internatonal Conference on Artfcal Intellgence, Modellng & Smulaton Comparng Image Representatons for Tranng a Convolutonal Neural Network to Classfy Gender Choon-Boon Ng, Yong-Haur Tay, Bok-Mn Go Faculty of Engneerng and Scence Unverst Tunku Abdul Rahman Kuala Lumpur, Malaysa {ngcb,tayyh,gobm}@utar.edu.my Abstract In ths work, we evaluated the effect of dfferent mage representatons on the classfcaton performance of a convolutonal neural network. Several dfferent methods for normalzaton of the nput data were also consdered. The network was dscrmnatvely traned for the task of gender classfcaton of pedestrans. A publcly avalable dataset was used for tranng, contanng both frontal and rear vews of pedestrans. The best result was obtaned usng grayscale representaton as compared to RGB and YUV, gvng crossvaldated accuracy of 81.5 % on the dataset. The performance of the convolutonal neural network s compettve and comparable to prevous works on the same dataset. Keywords- convolutonal neural network; pedestran; gender classfcaton; nput representaton I. INTRODUCTION A convolutonal neural network s a class of neural network wth a multple-layered archtecture and usng the dea of shared flter weghts to learn task-dependent features from the tranng data provded to t. Its orgn s nspred bologcally by the herarchcal nature of the mammalan prmary vsual cortex. Other such bologcally nspred network archtectures whch have been studed n computer vson system nclude the Neocogntron [1], whch can be consdered as ts predecessor, and the HMAX model [2]. Neural networks wth several hdden layers such as convolutonal neural networks are consdered to have a deep archtecture. Compared to shallow archtectures, a deep archtecture s beleved to be able to more effcently learn complcated functons to represent the hgh-level abstractons requred for complex tasks n artfcal ntellgence [3]. Untl recently, wth the breakthrough of greedy layerwse tranng ntroduced n [4], t has been dffcult to tran such multlayered network archtectures. However, the convolutonal neural network has been a notable excepton, havng been successfully traned usng back-propagaton [5]. Convolutonal neural networks have acheved hghly compettve results n a varety of pattern recognton problems such as handwrtten dgt recognton [6], face detecton [7], traffc sgn classfcaton [8] pedestran detecton [9], object recognton [10] and acton recognton [11]. A convolutonal neural network ntegrates feature extracton and classfcaton nto a sngle framework. Dscrmnatve feature detectors are learnt automatcally by the network from the data durng the supervsed tranng stage, hence the classfer system desgn does not need to rely on dffcult hand-engneered feature extracton. We are nterested n the problem of gender classfcaton of humans usng computer vson. The majorty of prevous research reported n lterature reled on usng only the face or head regon alone to tran a gender classfer [12]. In ths work, we make use of the whole body of a person to tran a gender classfer. Ths s justfable n stuatons where the face s of nsuffcent resoluton or not vsble (e.g. rear vew of the person) The problem of nferrng gender from whole body nformaton was frst nvestgated n [13], n whch Hstogram of Orented Gradents (HOG) features [14] were used to represent patches of the human body mage. These features were used to tran a boostng type classfer. HOG features have also been used n other pror works [15][16]. Gabor flters were used to extract features n [17]. The features dmensons were reduced usng varous manfold learnng methods such as Prncpal Component Analyss and Localty Senstve Dscrmnant Analyss. The features are then fed nto a lnear Support Vector Machne classfer. The best result was obtaned n a ppelne contanng a vew classfer before the gender classfer. In ths work, we traned a convolutonal neural network for the task of pedestran gender classfcaton. When used for such vson tasks, the nput data fed nto the network are the raw pxel values of the mage. An mage can be represented n varous color models. Varous researchers have used dfferent mage representatons to tran the convolutonal neural network, such as grayscale, RGB and YUV. However, t s not clear whch s the most sutable representaton for our task. Referrng to the pror works on recognzng gender from whole body, color was used as supplementary features n [15] [16]. The remander of ths paper s organzed as follows. In secton 2, the methodology of our experment s presented. In secton 3, detals of the experments and tranng procedures are gven. The results are analyzed and compared n secton 4. Fnally, secton 5 presents the concludng remarks for our paper and future work s suggested /13 $ IEEE DOI /AIMS

2 II. METHODOLOGY A. Convolutonal Neural Network We use a dscrmnatvely-traned convolutonal neural network, nspred by the work of LeCun [6] for our experment. A convolutonal neural network typcally ncludes two dfferent knds of layers correspondng to the smple and complex cells n the vsual cortex model of [18], performng local feature detecton and output poolng respectvely. Each layer s comprsed of a stack of so-called feature maps, whch are obtaned by performng convoluton or subsamplng on the prevous layer s feature maps. The archtecture of a convolutonal neural network attempts to acheve some degree of scale, translaton and deformaton nvarance n ts structure through the use of local receptve felds, shared weghts and spatal subsamplng. Fg. 1 shows the archtecture of the convolutonal neural network that we have used n our experments. The network contans of a total of 7 layers, ncludng the nput and output layers. The hdden layers consst of 2 stages of convoluton and subsamplng layers, labeled as C and S respectvely and a fully-connected layer of perceptron unts labeled as F. The feature maps n the frst convoluton layer C1 are obtaned as a result of the convoluton operaton of a set of flters wth the nput unts, whch are then passed through a squashng actvaton functon. Let W,j be the flter of sze f g whch connects the -th feature map from the prevous layer I to the j-th feature map C j and b j the correspondng tranable bas. The feature map s obtaned as follows, C j ( W, j S I b Here denotes the convoluton operaton and S denotes the set of all or selected feature maps from the prevous layer. We use the hyperbolc tangent functon as the squashng actvaton functon whch ntroduces nonlneartes, as gven by the followng equaton, x x e e ( x) tanh( x) (2) x x e e Gven that the sze of a feature map s hw, convoluton wth flter of sze f g wll produce an output feature map wth a sze of (h f + 1) (w g + 1),.e. convoluton s only performed on vald pxel values as opposed to full convoluton whch requres zero-paddng pxels of the mage borders. The same set of weghts for the flter s shared by each unt of a feature map. The values of these weghts are parameters whch are obtaned by supervsed tranng. Hence, weght sharng greatly reduces the number of tranable parameters n order to help reduce overfttng. Overfttng s a problem n whch a traned classfer performs well on the tranng data but does poorly when presented wth new data whch are not from the tranng set. Such a classfer s sad to have poor generalzaton ablty. j ) (1) Next, the feature maps of the subsamplng layer S2 s obtaned by downsamplng each feature map n layer C1. We use the maxmum-poolng operaton [2], n whch the largest value n a local regon s taken. Specfcally, each feature map n layer C1 s parttoned nto non-overlappng pp subregons. The largest value s output from each sub-regon to become the unts that form the feature map of the S2 layer. Every feature map s thus downsampled n ts sze by a factor of p to form the next layer S2. Fg. 2 llustrates an example of the maxmum-poolng operaton. The 66 mage, after applyng 22 maxmum-poolng, s reduced to 33. In the 22 sub-regon shown, the largest value s taken for the subsampled output. Smlarly, the convoluton layer C3 s obtaned followed by subsamplng layer S4. In our work, each feature map n layer C3 s connected to all the feature maps from layer S2. Layer F5 s a layer of perceptrons smlar to the hdden layer of a neural network, each unt fully connected to all the unts n the feature maps of layer S4. The unts n layer F5 also use hyperbolc tangent actvaton functon. Fnally, the output layer unts are connected to all the unts of layer F5. Softmax actvaton unts are used n the output layer to form a lnear classfer. The unt apples a softmax functon on the nput x, gven the weghts W m and bas b m of the m-th unt. Gven n s the number of unts, the output of the m-th unt s gven by the followng equaton, y x m ( ) n exp( W x b 1 exp( W x b ) Each softmax actvaton unt s assocated wth a class label and the output value gves the probablty of the class gven the nput, weghts and bas. The largest output probablty gves the predcted class. Fg. 1. Archtecture of convolutonal neural network. Fg. 2. Example llustratng maxmum poolng operaton. m m ) (3) 25

3 B. Image Representatatons The nput data whch are fed nto the network are the raw pxel values of the mage. An mage can be represented n varous dfferent color models. In our experments we consder three dfferent representatons grayscale, RGB, and YUV. Grayscale representaton contans only ntensty or lumnance nformaton. A grayscale mage, assumng pxel depth of 8 bts, contans pxel values from 0 to 255, presentng a sngle channel as the nput to the neural network. In the RGB model, an mage conssts of three component mages, one for each of the prmary colors red, green or blue. Thus, there are three channels to the nput of the neural network. The YUV model separates a color mage nto lumnance (Y) and chromnance (U and V) components. YUV s actually a famly of color spaces, but generally, n computer vdeo YUV usually refers to the Y C b C r color space [19] where C b and C r are the color dfference values obtaned by subtractng luma Y from blue and red components respectvely In our experments, we use Y C b C r to represent the YUV model. The relatonshp between Y C b C r and RGB color models are gven by the followng equatons, Y = R G B (4) C b = B Y (5) C r = R Y (6) C. Dataset The MIT Pedestran dataset [20] was used n the experments. The dataset conssts of 924 color mages of people. Each mage s 64x128 pxels, contanng the whole body of a person algned to the center of the mage. We used the ground truth for gender label as provded by [13], consstng of 600 males and 288 females. There are vews of both front (420 mages) and rear (468 mages) of a person. The mages were then cropped to 54x108 by removng equally the border pxels and then reszed down to 40x80. Generally, from our ntal work, we obtaned better results usng cropped mages compared to the orgnal. Examples of the cropped mages from the dataset are shown n Fgure 3. Fg. 3. Example of cropped mages from the MIT Pedestran dataset [20] Three dfferent mage representatons as mentoned n the prevous secton were used n the experments. The orgnal mages n RGB are converted to grayscale and YUV. Addtonally, we appled three dfferent normalzaton methods. The value of the pxels n each channel of an mage are scaled down to the range [0,1] or [-1,1], whch we refer to as Type I and II respectvely. Specfcally, for every pxel x n an mage, Type I: x x / 255 (7) Type II: x ( x )/127.5 (8) In the thrd method, Type III, the pxel values are normalzed to zero mean and unt standard devaton. nput data whch are fed nto the network are the raw pxel values of the mage. Gven an mage, the mean and standard devaton 2 of the pxels are calculated. The followng s then appled to every pxel x, Type III: x ( x )/ 2 (9) III. EXPERIMENTS We used the followng parameters for the archtecture of the convolutonal neural network. The sze of the nput mage s 40x80. The number of nput channels s one for grayscale mage and three for color (RGB and YUV) mages. Flters of sze 5x5 are used to obtan 10 feature maps of convoluton layer C1, resultng n feature maps of sze 36x76. Maxmum-poolng operaton s appled to 2x2 nonoverlappng patches of each feature map, resultng n subsamplng layer S2 wth feature maps of sze 18x38. Layer C3 contans 20 features maps wth the sze 14x34 produced from 5x5 flters, each connected to all the feature maps n layer S2. Layer S4 contans feature maps of sze 7x17 obtaned after 2x2 maxmum- poolng operaton. Layer F5 contans 25 neuron unts whch are fully connected to layer S4. The output layer has two softmax actvaton unts for bnary classfcaton. All unts use hyperbolc tangent actvaton. The total number of tranable weghts ncludng bases s 64,857 when usng grayscale mage as nput and 69,857 when usng color mages. For the supervsed tranng of the network, the weghts were ntalzed randomly from a unform dstrbuton n the range of [-(6/f), (6/f)]. Followng the suggeston n [21], f equals the number of nput connectons plus the number of output connectons. The weghts were updated by backpropagaton usng mn-batch stochastc gradent descent on the tranng dataset wth randomzed order. A small batch sze of 4 was chosen, and we tred varous learnng rates unformly dstrbuted between 0.01 and 0.2. Fve-fold cross-valdaton method was appled as follows. The dataset was dvded nto fve folds, wth one fold held out as the valdaton set whle the rest formed the tranng set. After each epoch, the accuracy of the classfer on the valdaton set was checked. The hghest valdaton set accuracy after 500 epochs was taken as the best result and the mean of the valdaton results was taken as the overall accuracy. 26

4 IV. RESULTS AND DISCUSSIONS Gven that three dfferent mage representatons were evaluated, each normalzed by three dfferent methods, as mentoned n the prevous secton, ths resulted n nne dfferent combnatons of datasets to tran the convolutonal neural network. The results obtaned are shown n Table 1, where the lowest average error from fve-fold cross valdaton s shown n each case. Overall, t can be observed that grayscale produces the lowest error, regardless of the type of normalzaton used. RGB s second best, gvng better results compared to YUV. In Fg. 4, the average error at varous learnng rates s plotted for the dfferent mage representatons wth Type I normalzaton. The average error ncreases by a large amount when the learnng rate becomes too large, and slghtly when too small. TABLE I. EXPERIMENTAL RESULTS Normalzaton Average error (%) method grayscale RGB YUV Unnormalzed Type I Type II Type III Fg. 4. Plot of average error for Type-I normalzaton Fg. 5. Plot of average error for grayscale representaton TABLE II. COMPARISON WITH OTHER METHODS Method Average accuracy std. dev. (%) Cao et al. [13] Collns et al. [15] a Guo et al. [17] Our classfer Usng frontal vew mages only It can also be observed that the best normalzaton method s Type I, n whch the data was scaled to the range [0,1]. It gves the lowest error for all three dfferent mage representatons. Indeed, some form of normalzaton seems essental for achevng good performance, as otherwse the average error s sgnfcantly hgher. In Fg. 5, the average error at varous learnng rates s plotted for grayscale representaton wth dfferent normalzaton methods. For dfferent learnng rates, Type II and III do not have such a large varance of error compared to Type I. Ths was also observed for the RGB and YUV representatons (charts not shown here). In any case, the lowest error s found at dfferent learnng rates. In terms of computaton tme, grayscale representaton, havng only one mage channel, has the advantage of slghtly faster tranng tme compared to RGB or YUV wth three mage channels. In our mplementaton usng Python wth the Theano mathematcal computaton lbrary [22] runnng on GPU, t takes an average of 4.8 mnutes for a sngle run of 500 epochs when usng grayscale compared to 5.1 mnutes usng RGB or YUV, approxmately 6% faster. The lowest error overall s 18.53% wth normalzaton to [0,1] and grayscale representaton. Ths corresponds to an accuracy of %. As shown n Table 2, the performance of the convolutonal neural network s compettve and comparable to other methods appled on the same dataset. To the best of our understandng, these methods also reported the average accuracy from fve-fold cross valdaton. V. CONCLUSION In ths paper, we have compared three dfferent mage representatons for tranng a convolutonal neural network to classfy the gender of pedestrans. Three dfferent normalzaton methods were used for preprocessng. From the experments, grayscale representaton produced the best result, followed by RGB and YUV. Usng grayscale mages normalzed to the range of [0,1] resulted n average gender classfcaton accuracy of 81.5% on the MIT pedestran dataset, whch s compettve wth prevous works on the same dataset. In the future, we plan to attempt more challengng datasets of people n varous poses and artculatons. ACKNOWLEDGMENT C.B.N. gratefully acknowledges the support obtaned from UTAR Research Fund and UTAR Staff Scholarshp Scheme. 27

5 REFERENCES [1] K. Fukushma, Neocogntron: A self-organzng neural network model for a mechansm of pattern recognton unaffected by shft n poston, Bologcal Cybernetcs, vol. 36, no. 4, pp , [2] M. Resenhuber and T. Poggo, Herarchcal models of object recognton n cortex., Nature Neuroscence, vol. 2, no. 11, pp , Nov [3] Y. Bengo, Learnng deep archtectures for AI, n Foundatons and Trends n Machne Learnng, vol. 2, no. 1, 2009, pp [4] G. Hnton, S. Osndero, and Y. Teh, A fast learnng algorthm for deep belef nets, Neural Computaton, vol. 1554, no. 18, pp , [5] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagaton appled to handwrtten zp code recognton, Neural Computaton, vol. 1, no. 4, pp , [6] Y. LeCun, L. Bottou, Y. Bengo, and P. Haffner, Gradentbased learnng appled to document recognton, Proceedngs of the IEEE, vol. 86, no. 11, pp , [7] M. Osadchy, Y. Cun, and M. Mller, Synergstc face detecton and pose estmaton wth energy-based models, The Journal of Machne Learnng Research, vol. 8, pp , [8] D. Cresan, U. Meer, J. Masc, and J. Schmdhuber, A commttee of neural networks for traffc sgn classfcaton, n Internatonal Jont Conference on Neural Networks, 2011, pp [9] P. Sermanet and K. Kavukcuoglu, Pedestran detecton wth unsupervsed mult-stage feature learnng, n Internatonal Conference on Computer Vson and Pattern Recognton, 2013, n press. [10] A. Krzhevsky, I. Sutskever, and G. Hnton, ImageNet classfcaton wth deep convolutonal neural networks, n Advances n Neural Informaton Processng Systems, 2012, vol. 25, pp [11] S. J, W. Xu, M. Yang, and K. Yu, 3D convolutonal neural networks for human acton recognton, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 35, no. 1, pp , [12] C. Ng, Y. Tay, and B. Go, Recognzng human gender n computer vson: a survey, n PRICAI 2012: Trends n Artfcal Intellgence, Sprnger Berln Hedelberg, 2012, pp [13] L. Cao, M. Dkmen, Y. Fu, and T. S. Huang, Gender recognton from body, n Proceedngs of the 16th ACM Internatonal Conference on Multmeda, 2008, pp [14] N. Dalal and B. Trggs, Hstograms of orented gradents for human detecton, n 2005 IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, 2005, vol. 1, pp [15] M. Collns, J. Zhang, and P. Mller, Full body mage feature representatons for gender proflng, n 2009 IEEE 12th Internatonal Conference on Computer Vson Workshops, 2009, pp [16] L. Bourdev, S. Maj, and J. Malk, Descrbng people: A poselet-based approach to attrbute classfcaton, n 2011 IEEE Internatonal Conference on Computer Vson, 2011, pp [17] G. Guo, G. Mu, and Y. Fu, Gender from body: a bologcally-nspred approach wth manfold learnng, n Proceedngs of the 9th Asan Conference on Computer Vson, 2009, no. 1, pp [18] D. Hubel and T. Wesel, Receptve felds, bnocular nteracton and functonal archtecture n the cat s vsual cortex, The Journal of Physology, vol. 160, pp , [19] About YUV vdeo, [Onlne]. Avalable: [20] M. Oren, C. Papageorgou, P. Snha, E. Osuna, and T. Poggo, Pedestran detecton usng wavelet templates, n Proceedngs of IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, 1997, pp [21] X. Glorot and Y. Bengo, Understandng the dffculty of tranng deep feedforward neural networks, n Proceedngs of the Internatonal Conference on Artfcal Intellgence and Statstcs, 2010, vol. 9, pp [22] J. Bergstra, et al. "Theano: a CPU and GPU math expresson compler," n Proceedngs of the Python for Scentfc Computng Conference,