LOCAL FEATURE EXTRACTIO AD MATCHIG METHOD FOR REAL-TIME FACE RECOGITIO SYSTEM Ho-Chul Shn, Hae Chul Cho and Seong-Dae Km Vsual Communcatons Lab., Department of EECS Korea Advanced Insttute of Scence and Technology, Daeeon, Korea e-mal : horse@sdvson.ast.ac.r ASTRACT Ths paper ntroduces a new local feature extracton and matchng method for the localzaton and classfcaton of frontal faces. Proposed method s based on the bloc-bybloc proecton onto drectonally classfed and quantzed egen-vertors n frequency doman, called as QCEs (Quantzed and Classfed Egen-locs. QCEbased feature extracton s effectve and relable comparng to smple zgzag scannng of DCT-coeffcents or PCA. Also proposed new bloc-based selectve matchng method s faster than exhaustve sldng wndow-based matchng methods and robust to partal mssng or occluson. Performances of proposed methods are verfed by the frontal face localzaton and classfcaton test. Especally, the test result shows that proposed method s utlzable for the mplementaton of a real-tme face recognton system. feature extracton method. PCA extracts most expressve feature and FLD extracts most dscrmnatve feature. FLD may seems to be more proftable than PCA n pattern recognton, but f the number of tranng mages are not enough, FLD can t warrant better performance than that of PCA. Le the egenfaces[4] or fsherfaces[5] method, PCA or FLD are generally utlzed to extract global features. ut these globalzed feature extracton cannot properly cope wth the partal changng, partal dstortons, mssng part, occluson and msalgnment. Moreover PCA and FLD need serously large computaton loads so that they are not proper to fast real-tme system. De : DCT-ed lco PI : loc Pattern Index LFV : Local Feature Vector CQE : Classfed & Quantzed Egen loc I (, (x,y I(x,y H W D-DCT. ITRODUCTIO Feature extractor n recognton system must be desgned to select dscrmnatve characters of target patterns and extract them robustly to the varous nose and dstorton. When we have a pror nowledge about features - what and where the effectve features are-, desgnng a feature extractor s easy and smply. ut n many applcatons, nowledge about features s not avalable and also fndng them s very hard wor, especally n face recognton. Many researchers have proposed the feature extracton method that can automatcally fnd effectve features and robustly extract them. Especally they have been concentrated on the methods whch extract the features from the statstcal nformaton of tranng mages by well nown PCA or FLD. PCA(Prncpal Component Analyss, FLD (Fsher Lnear Dscrmnants are statstcal lnear-mappng method most wdely used as an generalzed automatc Drectonal bloc-pattern Classfcaton PI Proecton onto Pattern- s CQEs De Swtchng Proecton onto Pattern- s CQEs Swtchng Proecton onto Pattern-L s CQEs ( PI (,, LFV (, Fg.. Proposed bloc-based feature extracton method D-DCT(Dscrete Cosne Transform[] and VQ(Vector Quantzaton[3] - based bloc-by-bloc feature extracton methods have been proposed to solve the problem of partal dstortons. In these methods, through the localzed normalzaton, they can reect or reduce the partal dstortons. Addtonally they have many advantages n mplementng the fast recognton system, because D-DCT or VQ needs comparatvely lower computatonal load than PCA or FLD and have
compatbltes wth varous mage codng methods and hardware structures. Man problem n desgnng a DCT or VQ-based feature extractor s how to arrange and select the coeffcents for the hgher performance. Most prevous methods select low frequency coeffcents wth zg-zag scannng or use code-vectors assocated wth a few low frequency components. ut for some localzed regon of mages whch contan more complex edges or textures, hgher frequency coeffcents are more valuable for the dscrmnaton than lower frequency components. In these senses, coeffcents selectng and arrangng methods should be vared by the localzed statstcal character of target pattern. In ths paper we ntroduce a new D-DCT-based feature extracton method and bloc matchng method. Showed n Fg., proposed feature extracton method frstly classfes blocs by ther drectonal characterstcs and extracts feature vectors usng proposed QCE (Quantzed and Classfed Egen-loc from blocs. QCE s a nd of bass vectors for proecton that are conssts of 3~0 weghtng-values for DCT-coeffcents. The weghtng-values came from quantzng the DCTcoeffcents-egen-vectors of the set of gathered same drectonal blocs. Though t requres small number of calculaton, wth QCE, a few weghted summng of coeffcents s more effectve than the smple zg-zag coeffcent selecton or bloc-level PCA[] because of selectve usng of optmal bass as ther drectonal characters. Drectonal pattern classfcaton of bloc s smple wor usng the rato of frst vertcal and horzontal DCTcoeffcents. ut through the pre-classfcaton of blocs, effectve representaton of bloc and fast bloc-based matchng s avalable. Ths fast matchng method s descrbed n chapter 5. Performances of proposed methods are verfed by face-localzaton and recognton test wth mages polluted by varous partal dstortons le an mssng, occluson and llumnaton change, etc. Also, comparson wth other global or local methods shows that our method s relable and fast so that s applcable to real-tme face recognton system. remander resulted from dvdng a by b. W and H are the wdth and heght of mage. for 0 u, 0 v and 0 W /, 0 H / ( u = α ( u I ( x, y β ( x, y, u, ( (, v y= x= / f t = 0 or = 0 where (, = t α t t / otherwse ( ( x + πt ( x + π t and β ( x, x, t, t = cos cos (3 for 0 u W and 0 v H C u, v ( rem( u,, rem( v, (4 ( ( quo( u,, quo( v,. DC-reecton and energy normalzaton DC component reecton and AC component energy normalzaton can reduce the effects of global and local llumnaton changes caused by lghtng condtons. Modfed ( and (4 equatons show these processng. (, α ( u ( u = y = x= I ( x, y β ( x, y, u u = 0 v = 0 (, ( u ( 0 f quo( u, = 0andquo( v, = 0 C ( u, v = (4 ( (,, (, ( (,, (, f (, 0or (, 0 quo u quo v rem u rem v quo u quo v 3. DIRECTIOAL CLASSIFICATIO OF LOCK In a bloc regon, drectons of edges or textures are one of the most dscrmnatve and nose-robust features. Especally, the rato of frst vertcal and horzontal DCTcoeffcents, (,0/(0,, drectly matches wth the drecton of edgeness n a bloc. The bloc can be classfed as ther drectonal characters by estmated drectons usng arctan ((,0/(0,. Proposed drectonal bloc classfcaton method n DCT-doman s presented by next equatons and pseudocode.. D-DCT. loc-by-bloc -pont D-DCT We dvde a gven raw mage, I(x,y, nto square blocs wth sze of and apply -pont D-DCT to each bloc. (, (u,v and C(u,v, can be obtaned by ( and (4. (, s the poston ndex of a bloc n a gven mage. Operatons - quo(a,b and rem(a,b means the quota and
e π/ θ L L- 3 L- T R Unrelable Pattern regon L- 3 L- e that t s applyng PCA on the DCT doman and selectve constructon of proecton bass accordng to the drectonal bloc pattern. Separated bass constructon by DCT and PCA can compact energy of bloc at a few selected postons n DCT doman. y quantzng these separated bass vectors, we can reduce the computatonal load wthout loss of effcency of energy compacton rates. Ths process s showed n Fg. 4 n detal. 4.. PCA n DCT doman Fg.. Drectonal classfcaton of bloc by angle of D D= (,0 + (0, (4 = (,0 (0, (5 D + ( (0,/ (,0 D= arctan (6 L = π/θ L + (7 < Drectonal bloc classfcaton algorthm> DC( DCT-ed bloc { f ( D T R, PI 0 (unrelable pattern else f ( 0.5* D < (,, PI L (mxed pattern else { f( D<0 D += π f( 0 D < θ L/ or π-θ L/ D < π, PI else f (θ L/ D < 3θ L/, PI else f ((L-3θ L/ D < (L-θ L/, PI L- } } return PI Proposed drectonal pattern of bloc conssts of π/θ L drectonal patterns, one mxed pattern and one unrelable pattern. In the process of classfcaton, when the (, s comparatvely large, the bloc wll be classfed as a mxed one whch means that t has more complex pattern than a sngle edge. If D s too small, the bloc wll be classfed as unrelable pattern ncludng bloc. They wll be excluded or reected durng the feature extracton or matchng process. locs of mrrored patterns whch have same drectonal characters but have nverted ntensty dstrbuton wll be classfed as same pattern. It s because that the localzed ntensty nverson s frequently occurred n mages by lghtng condton change. 4. QCE Operaton means raster scannng of bloc to mae a matrx to a sngle column vector and " s nverse one. DC(, functon for drectonal classfcaton of bloc ntroduced n prevous chapter, returns the ndex of bloc pattern. S s set of DCT-ed blocs gathered through all tranng mages and S, S,, S L are the classfed sets of tranng blocs by drectonal patterns to L. S = { R,..., M } ( = S = S S... S L and ( S = { DC ( =, S, =,..., M } (3 ext, apply PCA to each pattern s tranng sets, gven S, S,, S L, for =,, L, M m =, Ξ = M ( m ( m M = M = get ( u, λ satsfes, where =,...,, T (4 Ξ u = λ u (5 λ λ... λ, u R and λ R, << " pattern- s egen-vectors = u =,..., } { (6 Last, quantzes each pattern s egen-vectors. gven T Q = Quantzaton level, T s = Decson level, T Q( u = [ u u... u ],T Q = {( l, c c= quo ( ul, TQ TS } (7 " pattern- s QCEs= Q ( u, T =,..., T } (8 { Q Fnally resulted QCEs consst of quantzed weght values matched at a few postons n DCT-doman and some examples of QCEs are showed as the form of nverse-dct-ed mages n Fg. 3. In ~5-th egen-blocs of each drectonal bloc patterns, we can notfy that ther drectonal characters are domnantly revealed. Proposed QCE(Quantzed and Classfed Egen-loc s new type of proecton bass for the dmensonalty reducton of mage bloc. Orgnal concept of bloc-bybloc PCA for feature extracton has been ntroduced as Egenpaxel method [], but QCE method s dfferent n
0< D < π/8 7π /8< D < π 3 4 5 6 7 8 9 0 Fg. 3. Examples of nverse-dct-ed QCEs (8x8-bloc sze, L= π/8< D < 3π/8 3π/8< D < 5π/8 5π/8< D < 7π/8 7π/8< D < 9π/8 9π/8< D < π/8 π/8< D < 3π/8 3π/8< D < 5π/8 5π/8< D < 7π/8 Mxed pattern Inverse DCTed mages of Classfed egen blocs DCT & KLT Gathered blocs for learnng whch have been classfed as a same drectonal pattern 00 0.63 05 0.773 09 0 03 06 7 03 0 3 0 0 00 00 04 00 08 00 00 00 00 00 0 00 00 03 00 00 00 00 Classfed Egen-blocs 0 05 00 0 00 00 00 00 0 00 00 00 00 0 00 00 0 0 00 00 00 00 00 00 0 00 00 00 00 00 00 00 Quantzaton Poston value (,0 0.8 (0, 0.6 Quantzed & Classfed Egen-blocs (QCEs Select non-zero weghts 0.6 0.8 Fg. 4. QCE (Quantzed and Classfed Egen-loc constructon process 4. Local feature extracton usng QCE If the pattern ndex of a gven bloc, (, (u,v whch comes from (,-poston n mage, s, then local feature vector(lfv s gven as z (, by proecton gven bloc onto pattern- s QCEs. T T U = [ u u... u ], z (, R T LFV (,= z = U m (9 ( (, (, Conceptually, feature extracton by proecton s (9, but real mplementaton of feature extracton usng QCE s ust weghted summng of a few DCT-coeffcents. Though QCE needs smaller or almost same-level computatonal load comparng to PCA, FLD or D-DCT, but acheves hgher representaton and dscrmnaton performance. 5. LOCK-ASED MATCHIG STRATEGY Proposed mage matchng method can be used for fndng translatonal poston of templates n nput mage or classfyng templates whch are ncluded n nput mages. Matched blocs between nput mage and templates are constraned as bloc-pars whch have same drectonal
pattern. Matchng measure (MD s calculated for the all matched blocs and accumulated at the pont of postonal dfference(pd n accumulaton map, A(m,n. To avod errors caused by msalgned bloc-matchng, templates are analyzed by overlapped blocs. Snce feature extracton from template s off-lne process, doubled feature extracton process wll not damage the performance of system serously. Resulted poston of maxmum value n A(m,n means the translatonal poston of template n nput mage. In the case of that many templates are ncluded n nput mages, we can set threshold value and select multple maxmum poston. Maxmum value tself means localzed smlarty between nput and template. Moreover, by comparng max-values of each template, the best matchng template can be decded. It can be utlzed as a classfcaton process. <Selectve bloc-based matchng algorthm > Reset the accumulaton map, A(m,n =0 where m=,-,-/,0,/,,, n=,-,-/,0,/,, For =0,,W/- and for =0,,H/- Get from nput mage. (, For =0,,W /- and for =0,,H /- Get ' from template. ( ', ' If DC( = DC( (, ' ( ', ' MD = K o e LFV (, LFV '( ', ' / σ PD = (, ( ', ' = ( ', ' A(PD += MD End for, Fnd maxmum pont (m*,n* n A(m,n Matchness A(m*,n* (0 Translaton (m*,n* ( 6. COMPUTATIOAL LOADS For the one bloc regon of sze, feature extracton usng proposed QCE wth fast-dct algorthm [8] needs about (log +3~0 calculaton unts. Though the fast- DCT and zgzag scannng method needs only log calculatons, but because of ts poor performance, comparatvely large dmensons of feature vector wll be needed. Large feature dmensons mae worse the speed of bloc matchng process. In the other hands, bloc-bybloc PCA needs 4 calculatons and also produce the large dmensonal feature vector for the same performance wth QCE. For the bloc-based matchng of two mages whch have K bloc regons, full search matchng method needs K mean square calculaton between local feature vectors are needed. ut proposed drectonal pattern-based matchng method needs approxmately K /L calculatons. L s the number of drectonal classes. If the dmenson of feature vector s smaller, calculatons for bloc matchng can be more reduced. The QCE-based feature extracton can extract smaller feature vectors comparng to smple zgzag scannng or bloc-level PCA because of ts good compacton abltes. 7. EXPERIMETAL RESULT 7. Face localzaton test All mages used n test came from Yale Face Database [5]. Face localzaton means to fnd the exact locatons of the face template n test mages. We assumed to have only one template mage and varous nput mages got under dfferent condtons were used for testng. In Fg. 7, test result shows that our method can fnd exact locaton of faces robust to changes of facal expresson, bacground texture and lghtng condton wth only one template. 7. Face classfcaton test In our face classfcaton test, only one template per class was used for classfcaton and we assume that there s only translaton between template and nput mage wthout rotaton or pose varaton of faces. Followed 5 dfferent methods were tested and compared. Manually gated one normal face mage per one person was used as a template for face classfcaton, showed n Fg. 5. Goal of classfcaton s to fnd exact personal dentfcaton for the orgnal 50 and generated 50 test mages of whch some parts are mssed (Fg. 6. Identfcaton of person s decded by comparng the matchness (eq.(0 between each templates and nput mage. Fg. 5. Examples of manually gated face templates Fg. 6. Examples of generated test mages Mss-classfcaton rate s showed n Fg. 8. Snce the general PCA (called as egenface[4] method cannot adapt to mssng parts or occlusons, almost all of the generated test mages are mss-classfed wth method. Wth the bloc-based localzed method; Method & 3, ncludng bacground and llumnaton changng damaged the classfcaton results. Method 4 wth localzed dstorton normalzaton processng showed comparatvely hgher performance but t cannot cope wth the ncluded nosy blocs came from bacground regon. Our proposed method showed best performance. It was well adaptng to mssng part by localzed bloc matchng and reect or mnmze the effect of ncludng
bacground regon by drectonal pattern-based selectve matchng. Moreover through the localzed normalzaton processng n DCT doman, our method sowed good performance under the effect of lghtng condton changes. "Template and selected relable blocs (blac-boxed Fg. 8. Face classfcaton test result - mss-classfcaton rate of varous feature extracton and matchng method Method Method Method 3 Method 4 Method 5 Feature extracton from one nput mage (30x40, Gray-pxel ms 64ms 4ms 36ms Matchng-based searcng for the one template and one nput mage 60ms 560ms (8-pxel step searchng 440ms 440ms 380ms 80~00 ms (Selectve matchng Fg. 9. Processng tme comparson Observed processng tme s lsted n Fg. 9. All methods were mplemented and run on the Intel Pentum-4.4GHz PC wth Mcrosoft vsual C++ 6.0. From the results of method through 4, the exhaustve full searchng matchng or full bloc matchng s not proper for the realtme system, but proposed selectve matchng method needs about 00ms and can be applcable to real-tme system that can process about 0 frames per second. The result of classfcaton test and processng tme observaton shows that proposed QCE-based feature extracton and selectve bloc-based matchng method can be the soluton for the accurate and fast real-tme face recognton system. 8. DISCUSSIO Fg. 7. Face localzaton test result( Left : selected relable blocs (blac-boxed n tested nput mages, Rght : Resulted locatons of template(whte-boxed Mss-classfcaton rate Rate(% 80 70 60 50 40 30 0 0 0 5 0 5 0 5 30 Dmenson of features extracted and used n a bloc regon Method. PCA wthout frst-3 + full search matchng* Method. loc-dct + zgzag + full bloc-based matchng** Method 3. loc-pca + full bloc-based matchng Method 4. method wth DC-reecton and AC-normalzaton Method 5. QCE + proposed selectve bloc-based matchng QCE-based local feature extracton and selectve bloc matchng method s adequate for fast real-tme system whch can do localzng and classfyng the occluded or partally polluted face mages. Performance of proposed method was verfed by test wth varous face mages. In our method, we assumed that target mages were already algned for rotaton or pose varaton, but n real condton, algnng problem s very mportant ssue. Advancng the method that can adapt to rotaton, pose and scale changes s nspred as a further wor. 9. REFERECES [] P. McGure and G. M. T. D Eleutero, Egenpaxels and a eural-etwor Approach to Image Classfcaton, IEEE Transactons on eural etwors, Vol., o. 3, May 00. [] C. Sanderson and K.K. Falwal, Polynomal features for robust face authentcaton, n Proceedng of 00 Internatonal Conference on Image Processng, Vol. 3, pp. 997-000, 00. * Full search matchng means sldng templates for all of postons n nput mages and calculatng smlartes. ** Full bloc-based matchng method matches a bloc n the template to the all of blocs n nput mage and accumulates smlartes for all poston n map.
[3] K. Kotan, C. Qu and T. Ohm, Face recognton usng vector quantzaton hstogram method, n Proceedng of 00 Internatonal Conference on Image Processng, Vol., pp. 05-08, 00. [4] Zhue, Y.L.Yu, Face Recognton wth Egenfaces, n Proceedng of the IEEE Internatonal Conference on Industral Technology, pp434-438, 994. [5] P.. elhumeur, J. P. Hespanha, and D.J.Kregman, Egen-faces vs. Fsherfaces: Recognton Usng Class Specfc Lnear Proecton, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 9, o.7, pp. 73-74, July 997. [6] A. K. Jan, R.P.W. Dun and J. Mao, Statstcal Pattern Recognton: A Revew, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol., o., Jan. 000. [7] abac Moghaddam and Alex Pentland, Probablstc Vsual Learnng for Obect Representaton, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 9, o. 7, 997. [8] Il Dong Yun and Sang U Lee, On the Fxed-Pont- Error Analyss of Several Fast DCT Algorthms, IEEE Transactons on Crcuts and Systems for Vdeo Technology, Vol. 3, O., Feb. 993. [9] D. S. Km and S. U. Lee, Image vector quantzer based on a classfcaton n the DCT doman, IEEE Transactons on Communcatons, Vol. 39, o. 4, 99.