VALIDATING DIRECTIONAL EDGE-BASED IMAGE FEATURE REPRESENTATIONS IN FACE RECOGNITION BY SPATIAL CORRELATION-BASED CLUSTERING

Transcription

1 VALIDATING DIRECTIONAL EDGE-BASED IMAGE FEATURE REPRESENTATIONS IN FACE RECOGNITION BY SPATIAL CORRELATION-BASED CLUSTERING Yasufumi Suzuki ad Tadashi Shibata Departmet of Frotier Iformatics, School of Frotier Scieces, The Uiversity of Tokyo 701 Kiba Bldg Kashiwaoha, Kashiwa, Chiba, , Japa ABSTRACT Directioal edge-based image feature represetatios have already bee developed ad applied to medical radiograph aalysis, face detectio, ad face idetificatio. The scheme of the feature-vector geeratio is cosidered as a dimesioality reductio from a high-dimesio edge feature map space to a low-dimesio edge histogram space. However, the validity of such schemes i dimesioality reductio has ot yet bee studied o the soud basis of statistics. I this paper, i order to validate the directioal edge-based feature represetatios, the scheme of dimesioality reductio employig the hierarchical clusterig based o the spatial correlatios of edges has bee ivestigated. I this study, the feature represetatios of images have bee evaluated usig facial images as the test vehicle. As a result, the validity of usig the directioal edge-based feature vectors i image recogitio has bee verified by the similarity betwee the results of the hierarchical clusterig ad the schemes employed i the feature represetatios. 1. INTRODUCTION The developmet of robust image recogitio systems is quite essetial i a variety of applicatios such as itelliget huma-computer iterfaces, security systems, ad so forth. For huma-like robust image perceptio, a image feature represetatio which extracts oly essetial features from images ad represets them i a vector format is of critical importace. A umber of feature represetatios have bee developed so far. I the visual cortex of aimals, it is kow that directioal edge iformatio i visual iputs plays a essetial role i early visual processig. Namely, biological systems rely o the spatial relatioship amog edges i various directios for image perceptio. Beig ispired by such a biological priciple, a directioal edge-based feature represetatio called Projected Pricipal-Edge Distributio (PPED for short) [1] has bee developed ad has bee successfully applied to had-writte patter recogitio ad medical radiograph aalysis [2, 3]. It has also bee applied successfully to the face detectio system [4] i cojuctio with aother directioal edge-based feature represetatio called Averaged Pricipal-Edge Distributio (APED for short) [5]. I additio, APED-like feature represetatios have bee applied to face idetificatio [6]. Both PPED ad APED feature vectors are geerated from the spatial distributios of the four directioal edges (horizotal, +45, vertical, ad 45 ). Sice the amout of data i the directioal edge maps is massive, the dimesioality reductio is carried out by producig the spatial distributio histograms from the four directioal edge maps. The feature-vector-geeratio schemes i PPED ad APED are compatible with the dedicated feature-extractio VLSI chip [7], makig the feature represetatio algorithms compatible to buildig real-time respodig systems. However, the schemes of dimesioality reductio i PPED ad APED have ot yet bee supported by statistical aalysis. The pricipal compoets aalysis (PCA) is ofte used for the dimesioality reductio of feature represetatios. The PCA calculates the orthogoal liear trasformatio which maximizes the variaces amog feature vectors. Such a liear trasformatio is derived from the eigevectors of the spatial correlatios i the image data. Although the PCA achieves the dimesioality reductio efficietly preservig the characteristics of images, the liear trasformatio of the PCA is ot suitable for direct hardware implemetatios due to the computatioal cost. The purpose of this paper to validate the directioal edgebased feature represetatios by the statistical aalysis. The dimesioality reductio i both PPED ad APED vector represetatios is carried out by takig the spatial histograms employig their ow partitios of the directioal edge maps. The dedicated VLSI chip has already bee developed [7] which is compatible to ay types of edge-histogram geeratio. Therefore, it is crucial to fid how to divide the directioal edge maps for geeratig histograms i order to maximize the performace i image classificatio. I this work, as like i the PCA, it is cosidered to maximize the variace amog feature vectors. For each directioal edge map, groupig the pixel sites havig the high correlatios together ito the same elemet of the feature vector icreases the variace as described i Sectio 3. I order to determie such partitios, the techique of hierarchical clusterig based o the spatial correlatio of the directioal edges has bee itroduced. I this work, the evaluatio has bee carried out usig a set of frotal facial images as a test vehicle. The results of the hierarchical clusterig are similar to the partitio employed i PPED or APED vector-geeratio scheme, thus validatig the directioal edge-based feature represetatios. 2. DIRECTIONAL EDGE-BASED FEATURE REPRESENTATIONS This sectio describes the directioal edge-based feature represetatios utilized for the face detectio system i the previous works [4, 5]. Firstly, the directioal edge-based feature maps which represet the distributio of four directioal edges are geerated. The, the 64-dimesio feature vectors are formed by takig the spatial histograms of the directioal 2007 EURASIP 1940

2 Horizotal +45-degree Vertical -45-degree Iput image Horizotal +45-degree Vertical -45-degree Figure 1: Directioal edge-based feature maps geerated from bright ad dark facial images. edge-based feature maps. 2.1 Directioal Edge-Based Feature Maps The first step i formig feature vectors is the geeratio of four feature maps i which edges are extracted from a pixel iput image i four directios. Figure 1 shows the relatioship betwee a iput image ad four feature maps. Each feature map represets the distributio of edge flags correspodig to oe of the four directios, i.e. horizotal, +45, vertical, ad 45. I the feature maps, each black pixel which we call a edge flag represets the locatio where a edge is idetified i the correspodig directio. These four directioal edge-based feature maps are regarded as represetig the most fudametal features extracted from the origial image. The procedure of feature-map geeratio is as i the followig. The iput image is firstly subjected to pixel-by-pixel spatial filterig operatios usig kerels of 5 5-pixel size. The kerel which gives the maximum value determies the directio of the edge at the pixel site. I order to retai oly edges of sigificace i feature maps, thresholdig operatio is itroduced. The threshold value is determied by takig the local variace of lumiace data ito accout. The itesity differeces betwee two eighborig pixels i the horizotal ad vertical directios i a 5 5-pixel block are obtaied ad the media of the 40 values of them is utilized as the threshold value for edge idetificatio. Thaks to such a thresholdig operatio, essetial edges represetig facial features are well extracted from both bright ad dark facial images as show i Fig. 1, thus makig directioal edge-based represetatios robust agaist illumiatio coditios. Details of the directioal edge-based feature maps are give i [1]. I the followig discussio, for each directio d H,P,V,M}, the directioal edge map is expressed as F d (x,y) 0,1}, where the directioal edge exists at the locatio (x,y) if F d (x,y) = 1. Here, H, P, V, ad M represet horizotal, +45, vertical, ad 45, respectively. 2.2 Directioal Edge-Based Feature Represetatios Although the directioal edge-based feature maps retai essetial feature iformatio i the origial image, the amout of data is still massive ad dimesioality reductio is essetial for efficiet processig of classificatio. Figure 2 illustrates the procedure of feature-vector geeratio i the Projected Pricipal-Edge Distributio (PPED) [1]. The feature vector i PPED is formed by projectig the feature maps alog the directio of the edges. I the horizotal edge map, for example, edge flags i every four rows are accumulated, Figure 2: Partitios of feature maps for vector geeratio based o projected pricipal-edge distributio (PPED). Horizotal +45-degree Vertical -45-degree Figure 3: Partitios of feature maps for vector geeratio based o averaged pricipal-edge distributio (APED). ad a spatial distributio of edge flags is represeted by a histogram. Similar procedures are applied to other three directios. Fially, a 64-dimesio vector is formed by cocateatig the four histograms. I the PPED feature represetatio, the iformatio of edge distributio alog the directio idetical to the directioal edge (e.g. the horizotal distributio of horizotal edge flags) is lost durig the accumulatio. I order to complemet such loss i PPED vectors, aother feature-vector geeratio scheme has bee developed [5]. I the Averaged Pricipal-Edge Distributio scheme (which was origially amed Cell Edge Distributio i [5]), every feature map is divided ito 4 4 cells each of which cotais -pixel sites as show i Fig. 3. The umber of edge flags i each cell is couted ad the umber costitute a sigle elemet i the vector represetatio by the APED scheme. A 64- dimesio feature vector i the APED scheme is obtaied by cocateatig these four vectors. 3. SPATIAL CORRELATION-BASED CLUSTERING The process of the feature-vector geeratio is cosidered as a dimesioality reductio i the space of four directioal edge maps oto the lower-dimesio spaces. I the feature-vector-geeratio schemes, each directioal edgebased pixel feature map of F H, F P, F V, or F M is divided ito groups as show i Figs. 2 ad 3. The umber of directioal edge flags withi each group is couted ad costitutes a sigle elemet i the feature vector, thus makig a -dimesioal feature vector. The dedicated VLSI chip [7] is capable of geeratig feature vectors usig such a scheme with ay form of feature-map partitios. Therefore, i the preset study, it is cosidered that how the feature maps should be divided for maximizig the variace amog the feature vectors geerated from facial images. I the fol EURASIP 1941

3 lowig discussio, pixel two dimesioal image data of a feature map are treated as a 4096-dimesio biary vector. Namely, each feature map F d (d H,P,V,M}) is coverted to a 4096-dimesio vector E d (i) which is expressed as E d (x + 64y) = F d (x,y), (1) where x = 0,1,2,...,63 ad y = 0,1,2,...,63. For each directio d H,P,V,M}, the partitio P d (i) of the feature map E d (i) is give as P d (i) j j N,1 j }, where i = 0,1,2,...,. This meas that the pixel site correspodig to the feature map E d (i) belogs to the P d (i)-th elemet i the feature vector. A mask fuctio M d (i, j) is defied from P d (i) as 1, if Pd (i) = j M d (i, j) = 0, otherwise, where i = 0,1,2,..., ad j = 1,2,3,...,. Note that the mask fuctio M d (i, j) satisfies the followig equatio: M d (i, j) = M d (i, j)} 2 = 1. (2) From the directioal edge map E d (i) defied i Eq. (1), a - dimesioal feature vector V d is geerated usig the mask fuctio M d as V d ( j) = M d (i, j)e d (i), (3) where j = 1,2,3,...,. Whe sample images are give, the mea E d (i) ad the variace Var(E d (i)) of the directioal edge-based feature maps E d (i) are obtaied as E d (i) = 1 Var(E d (i)) = 1 Ed k (i), (4) E k d (i) } 2 E d (i)} 2, (5) respectively, where Ed k (i) is the edge map of the directio d obtaied from the k-th face sample. The covariace C d (i, j) betwee two differet pixel sites E d (i) ad E d ( j) i the feature map is also obtaied as follows: C d (i, j) = 1 E k d (i)ek d ( j) E d(i) E d ( j). (6) The feature vector V d ( j) is geerated from the directioal edge map E d (i) usig Eq. (3). The mea V d ( j) ad the variace Var(V d ( j)) of the j-th elemet i the feature vector V d are expressed as V d ( j) = 1 Var(V d ( j)) = 1 Vd k ( j), (7) V k d ( j) } 2 V d ( j)} 2, (8) respectively, where Vd k ( j) is the j-th elemet i the feature vector i the directio d geerated from the k-th sample. From Eqs. (3), (4), ad (7), the values of Vd k( j)}2 ad 2 V d ( j)} are obtaied as V k d ( j) } 2 = 2 V d ( j) + M d (l, j)e k d (l)m d(m, j)e k d (m) M d (i, j)e k d (i) } 2, (9) } 2 = M d (i, j)e d (i) = 2 + } 2 M d (l, j)e d (l)m d (m, j)e d (m) M d (i, j)e d (i)} 2, (10) respectively. The value of Var(V d ( j)) is simplified usig Eqs. (5), (6), (8), (9), ad (10) as Var(V d (J)) = 2 M d (i, j)} Var(Ed (i)) + 2 M d (l, j)m d (m, j)c d (l,m).(11) Here, we cosider the summatio of variaces of featurevector elemets W d which is give as W d = Var(V d ( j)). From Eqs. (2) ad (11), W d is expressed as i the followig: W d = 2 + M d (l, j)m d (m, j)c d (l,m) Var(E d ( j)). (12) I Eq. (12), sice Var(E d ( j)) is a costat idepedet of the mask fuctio M d (i, j), maximizig W d is equivalet to maximize α d defied as α d = l= M d (l, j)m d (m, j)c d (l,m). (13) α d i Eq. (13) meas the summatio of all covariaces betwee ay pair of pixels which belog to the same elemet i the feature vector. I order to determie the divisio of four feature maps which maximizes the sum of variaces of feature-vector elemets (this is idetical to maximizig the value of α d ), the hierarchical clusterig techique has bee itroduced. All sites withi pixel widow are divided ito clusters usig the agglomerative hierarchical clusterig algorithm [8] where the covariaces betwee differet pixel sites are utilized as the dissimilarity measure EURASIP 1942

4 Horizotal +45-degree Vertical -45-degree Horizotal +45-degree Vertical -45-degree Complete likage Figure 4: Mea images of directioal edge-based feature maps geerated from 900 frotal facial images. Average likage Wd Summatio of variaces of vector elemets 1.6e+4 1.4e+4 1.2e+4 1.0e+4 8.0e+3 6.0e+3 4.0e+3 2.0e+3 0.0e+0 Horizotal +45-degree Vertical -45-degree Feature map PPED APED Feature represetatio Figure 5: Summatio of variaces of all feature-vector elemets for PPED ad APED. 4. EXPERIMENTAL RESULTS AND DISCUSSION I order to evaluate the variaces of feature-vector elemets i PPED ad APED, a statistical aalysis has bee doe as i the followig. I this work, 900 frotal face samples (100%, 80%, ad 60% scaled faces) from HOIP face database [11] were utilized for the evaluatio. For each of facial images, the directioal edge-based feature vector was geerated. The mea images E d (i) of 900 directioal edge-based feature maps are preseted i Fig. 4. The, for each directio (horizotal (H), +45-degree (P), vertical (V ), ad -45-degree (M)), the value of W d (d H,P,V,M}) which is the summatio of the variaces of feature-vector elemets was calculated. The values of W d i PPED ad APED are show i Fig. 5. I Fig. 5, the values of feature map represet the summatio of variaces of all pixel sites i the feature map, ad are equivalet to the value Var(E d( j)) i Eq. (12). The differece betwee the value of PPED ad that of feature map idicates the icremet of variaces α d caused by the dimesioality reductio durig the PPED-vector geeratio. The hierarchical clusterig was carried out o the covariace matrices derived from the 900 frotal face samples to determie the partitios of edge maps maximizig the variace. Figure 6 demostrates the results of hierarchical clusterig employig the complete likage algorithm [9] ad the average likage algorithm [10]. The clusterig result by the complete likage algorithm, which has the tedecy of makig clusters compact, shows that the eighborig pixel sites alog the directio of edge flags (e.g. horizotal eighborig sites of horizotal edge flags) are likely to be classified ito the same cluster. Such clusterig is very similar to the feature Figure 6: Results of agglomerative hierarchical clusterig of all pixel sites i directioal edge-based feature maps. Each feature map is divided ito clusters which are idicated by differet colors. Wd Summatio of variaces of vector elemets 8e+4 7e+4 6e+4 5e+4 4e+4 3e+4 2e+4 1e+4 0e+0 Horizotal +45-degree Vertical -45-degree PPED APED Complete likage Average likage clusterig clusterig Feature represetatio Figure 7: Summatio of variaces of all feature-vector elemets for each vector-geeratio scheme. map divisio scheme employed i the PPED vector geeratio show i Fig. 2. O the other had, it is iterestig to see the clusterig result by the average likage algorithm. It displays more coectivity tha that by the complete likage algorithm, i which each facial part such as the eyes, the ose, ad the outlie of a face is assiged to the same cluster. This implies that the feature map divisio scheme employed i the APED vector geeratio (Fig. 3) is essetial for facial image recogitio. The results of the clusterig suggest that PPED ad APED are essetial for facial image classificatio. However, PPED ad APED have bee developed as geeral-purpose feature represetatios. Therefore, we ca expect that employig the results of the hierarchical clusterig as the partitios ca improve the performace of facial image recogitio. Figure 7 shows the values of W d i Eq. (12) employig the results of hierarchical clusterig illustrated i Fig. 6 as the partitios of the edge maps are demostrated i compariso with those of PPED ad APED. The clusterig-based feature represetatios achieve larger variaces tha PPED ad APED. The prelimiary experimets of face detectio were carried out usig a image i CMU test set C [12]. I the template matchig-based face detectio, a pixel image was take from a iput image ad a feature vector was geerated from the take image. The, the feature vector was 2007 EURASIP 1943

5 900 frotal facial images as test samples, four directioal edge maps were aalyzed employig the hierarchical clusterig based o the spatial correlatios of edge flags. As a result, the validity of usig PPED ad APED feature vectors for facial image recogitio has bee verified by the similarity betwee the results of the hierarchical clusterig ad the partitio employed i PPED or APED vector-geeratio scheme. I additio, it has also bee demostrated that the hierarchical clusterig-based feature represetatios improve the performace of face detectio. REFERENCES Figure 8: Result of face detectio from a image i CMU test set C [12] usig PPED ad APED feature vectors. Figure 9: Result of face detectio usig hierarchical clusterig-based feature vectors. matched with all template vectors of face ad oface samples, ad classified as a face or a oface accordig to the category of the best-matched template vectors. This classificatio was carried out by pixel-by-pixel scaig the etire image. The 900 facial images ad 2000 radomly chose backgroud images were utilized for face samples ad oface samples, respectively. I the experimet, the cocept of multiple-clue criterio [5] has bee itroduced. Namely, oly pixel sites where both feature vectors classified as a face were determied as a true face. The results of face detectio usig PPED ad APED feature vectors are show i Fig. 8. Although all faces are detected, may false positives also occur. Figure 9 demostrates the result of face detectio usig the feature vectors utilizig the results of the complete likage clusterig ad the average likage clusterig show i Fig. 6. The umber of false positives has bee reduced without missig the true faces. (These false positives ca be elimiated by the facial parts verificatio algorithm described i [4].) This result implies that the directioal edge-based feature represetatios usig the hierarchical clusterig improve the performace of image recogitio. 5. CONCLUSIONS The directioal edge-based image feature represetatios have bee studied o the basis of statistic aalysis. Usig [1] M. Yagi ad T. Shibata, A image represetatio algorithm compatible with eural-associative-processorbased hardware recogitio systems, IEEE Tras. Neural Netw., vol. 14, pp , Sept [2] M. Yagi, M. Adachi, ad T. Shibata, A hardwarefriedly soft-computig algorithm for image recogitio, i Proc. EUSIPCO 2000, Tampere, Filad, September , pp [3] M. Yagi ad T. Shibata, Huma-perceptio-like image recogitio system based o the associative processor architecture, i Proc. EUSIPCO 2002, Toulouse, Frace, September , pp [4] Y. Suzuki ad T. Shibata, Multiple-Resolutio Edge- Based Feature Represetatios for Robust Face Segmetatio ad Verificatio, i Proc. EUSIPCO 2005, Atalya, Turkey, Sept [5] Y. Suzuki ad T. Shibata, Multiple-clue face detectio algorithm usig edge-based feature vectors, i Porc. of Itl. Cof. o Accoustics, Speech ad Sig. Proc. (ICASSP), Motreal, Caada, May , pp [6] Y. Suzuki ad T. Shibata, Illumiatio-ivariat face idetificatio usig edge-based feature vectors i pseudo-2d hidde Markov models, i Proc. EUSIPCO 2006, Florece, Italy, Sept [7] H. Yamasaki ad T. Shibata, A real-time imagefeature-extractio ad vector-geeratio VLSI employig arrayed-shift-register architecture, i Proc of Europea Solid-State Circuits Cof. (ESSIRC), Greoble, Frace, Sept , pp [8] W. H. E. Day ad H. Edelsbruer, Efficiet algorithms for agglomerative hierarchical clusterig methods, Joural of Classificatio, vol. 1, pp. 7-24, Dec [9] D. Defays, A efficiet algorithm for a complete lik method, Computer Joural, vol. 20, pp , [10] E. M. Voorhees, Implemetig agglomerative hierarchic clusterig algorithms for use i documet retrieval, Iformatio Processig ad Maagemet, vol. 22, pp , [11] Softpia Japa, Research ad Developmet Divisio, HOIP Laboratory, HOIP facial image database, catalog/top.html [12] H. Rowley, S. Baluja, ad T. Kaade, Neural etworkbased face detectio, IEEE Tras. Patter Aal. Machie Itell., vol.20, pp , Ja EURASIP 1944