Ne Knoledge-ased Face Image Indexng System through the Internet Shu-Sheng La a Geeng-Neng You b Fu-Song Syu c Hsu-Me Huang d a General Educaton Center, Chna Medcal Unversty, Taan bc Department of Multmeda Desgn, Natonal Tachung Insttute of Technology, Taan d Department of Management Scence, Natonal Tachung Insttute of Technology, Taan a ssla@mal.cmu.edu.t b gny@ntt.edu.t c s1894111@ntt.edu.t d hmhuang@ntt.edu.t bstract Image ndexng s one of the most popular research areas of computer vson, such as for dgtal lbrares, multmeda systems and TV program systems. Ths paper focuses on the face mage ndexng applcatons. The authors proposed a ne scheme for face mage ndexng by usng a knoledge-based method that artsts dra human portrats. The basc dea s to analyze artsts ho to dra a portrat based on human face proportons related knoledge. nd then, e use these rules to process face mage ndexng. Ths process provdes a ne alternatve ay for ndexng human face mage n terms of the knoledge-based method. Our system s composed of to man steps. The frst step s facal features extractng. nd then, t goes to human face mage ndexng. There are to advantages of ths system. Frst, the system provdes a smple synthess procedure. Secondly, the paper proposes an easy method hch s mportant for reducng atng tme of delayng cost of Internet server. Keyords face mage, ndexng, Multmeda database 1. Introducton In recent years, there has been sgnfcant progress n the human faces recognton. The task of searchng and even recognzng faces n a database, a computer seems to be more cost effect than a human observer does. Face ndex and recognton has a large number of research reports, nclude geometrc feature-based methods, template-based method [1], and more recently model-based methods [2,, 4, 5]. These technques are alays bult on a foundaton of mathematcal probablstc formulatons. Lttle attenton has been gven to the human ntuton and analyss. ased on these reports so far, fe researchers seem to have tred ths experment. Therefore, ths study attempts to use a human perspectve. We propose an approach hch combnes human pont of ve th model-based technology. Our system could be separated nto clent and server to parts. Fgure 1 shos the archtecture of our proposed system. On the clent, e use a tranng set of facal mages to buld a statstcal model and take the model to help user that extract facal features from an nput mage. On the server, the facal data that from the clent are processed by the relatonal ratonsof artst s knoledgebased method to further ndex the face mages. Fgure 1. The archtecture of our proposed system 2. Related Works The dffculty n understandng faces comes from the large degree of varablty possble n mages of any face. The follong three methods are most popular appled n most face mage ndexng system. 2.1 Geometrc feature-based method These studes have focused on the geometry features of a face. Usng the dgtal mage processng functon (such as detect edge) to get boundary of object (.e. face lne/bros) from an nput face mage. Then, t can base on the gotten boundary to search a full face. For nstance, Shh et al. [6] developed an automatc extracton of facal features. Geometrc feature-based method s especally good at detectng errors. Usng geometrc face method, there are geometrc features beng extracted such as eyes th eyebros, nostrls, and mouth, as shon n Fgure 2.
Fgure 2. n example of expermental results of Shh s developed system. 2.2 Template-based method The technque has been ntally proposed by Vola [1]. For a start, a classfer s traned th a fe hundreds of sample ves of a face, called postve examples, hch are scaled to the same sze (lke 20x20). fter tranng a classfer, t can be appled to a regon n an nput mage as shon n Fgure. The classfer outputs a "1", f the regon s lkely to sho the face. Otherse, the result should be denoted "0". Fgure. The proposed method by Vola. 2. Model-based method In order to recognze face mages, t s mportant to have facal features data. It s knon that faces can vary dely, but features can be grouped under nto to parts: n shape and texture. These are many model-based methods under ongong for research. Recently, some researchers proposed a poerful method to generate models. These models, such as ctve ppearance model (M) by hlberg [2], and Support Vector Machne (SVM) by Shh and Lu [], can be used to get shape and texture of facal features of face mages. Face mages usng the model parameters for classfcaton, t can obtan good results for person dentfcaton and expresson recognton usng a tranng and test set of stll mages. The basc dea s to adopt a statstcal approach, learnng the ays n hch the shape and texture of facal feature ponts of face mages. For nstance, f e are nterested n faces th expressons, e should choose set of tranng mages. The set should mages of people smlng, fronng, nkng, and so on. nd then, e labeled th a set of ponts defnng the facal features n each face mage. These ponts defne expressons of a face mage. Fgure 4 shos a set of 58 ponts used to label a face by usng the M-PI [5]. Fgure 4. Example of 58 ponts defnng facal features These proposed methods have been used to analyze facal feature ponts and generalze a model. nd then, e can get facal feature ponts by the ay of fndng the mnmum dfference beteen the nput mage and the generalzed model. These model-based methods are popular approaches n extractng facal feature.. Facal Feature Extractng Features of the faces hch t s desred to detect and locate are collected n advance n a face feature database. When the system detects human faces n the nput mage, t compares the features th those n the database and dentfes the faces. To ndex the human face mage, e requre a set of facal feature ponts. In ths study, e determne 19 facal feature ponts that demand for our proposed method. nd then, use the M-PI [5] that C++ mplementaton of the M frameork to buld a facal appearance model and take the model to help users to extract the nput mage. In order to extract these facal feature ponts, a human face s decomposed nto 19 facal feature ponts. They are to eyes (left and rght), a nose, a mouth, and the face lne. The result of the extractng processes s shon n Fgure5. Fgure 5(b) shos the nput mage. fter extractng facal features labeled by Fgure 5(a), the result of nput mage s extracted by usng M as shon n Fgure 5(c). Left Eye Rght Eye Nose Mouth Face Lne 1 2 4 5 0 Facal Features label (a)
here R R denotes the absolute value of R R. (b) 1 2 4 5 Fgure 5. (a) M for facal features extracton (b) Input mage; (c) The result of the face feature extracton by usng an M (c) We no take the Fgure 5(b) as an example. Fgure 7 shos the Top of the most lkeness mages on the Yale database. Top shos the former places of the result sequence n order correspondng to the Fgure 5 mage on the Yale Face Database. 4. Knoledge-ased Face Image Indexng fter the feature extractng processes, e can get the facal feature ponts from the clent. These feature ponts are used to search the possble mage from a human database on the server. For example, 1 means the dth of left and rght eyes. Moreover, means the dth of nose, and so on. For ndexng a human face, e use the relatonal ratos from a carcature textbook [7]. These ratos are as shon n Fgure 6. / 1 2 / / / 4 5 / 6 / 7 / h / h 1 h 2 / h h / h TOP1 TOP2 TOP Fgure7. Top of the most lkeness mages on the Yale database from Fgure 5(b) by our method. In a summary, the procedures of the system are: 1. gven face should extract facal feature ponts frst. The system apples an M approach to extract 19 feature ponts. 2. The system ll calculate the 10 relatonal ratos based on facal feature ponts.. nd then, the system use these feature ponts and relatonal ratos to search for canddate faces nput mage usng facal features n the server. 4. The system ll lst the top canddate faces based on lkeness degrees. Fgure 6. The relatonal ratos defned for ndexng In Fgure, these relatonal ratos descrbe the proporton of the face devoted to a partcular facal feature. For example, 4 / refers to the rato of the dth of the head to the dth of the mouth [8]. In general, these ratos have been used by artsts n drang a human portrat. In our method, e not only compare the dth values (such as and h values), but also use these feature ratos to calculate the lkeness degrees among the nput face mage and all face mages n the database. In mage ndexng systems, a varety of smple smlarty measures are used. The choce for one smlarty measure or another s generally drven by an expermental comparson on a labeled database. For example, t s knon that the total number of relatonal ratos alloable n the study s n and the probablty of feature rato, 0 n. Suppose the feature rato from an nput mage s denoted by R and R s the feature rato from the database. The lkeness degrees denoted by S can be calculated by 5. Expermental Results In order to test the accuracy of the ndexng system descrbed, a publc face database as used. These mages take from the Yale Face Database [9] that s avalable for donload from http://cvc.yale.edu. It ncludes 90 face mages hch s 15 ndvduals th 6 facal expressons mages. Our system frst locates 19 facal features from an nput mage. Then, the system apples the artsts knoledge rules to fnd a human face from the Yale database. Table 1 shos the results of ths study. If the nput mage can be ht Top of lkeness degrees correctly from the total number of the database, the results denote Pas. Otherse, e cal t false. Fnally, the result of successful rato s about 80% as shon n Table 1. S (1 ( n R R mn(1, R )) / n) 100%, 1
Table 1. The ndexng results of our system Subject01 Subject02 Subject0 Subject04 Subject05 Subject06 Subject07 Subject08 Subject09 Subject10 Subject11 Subject12 Subject1 Subject14 Subject15 Ht at TOP of lkeness degrees Fal Fal Fal Note: We defne the successful rato as Successful rato = pass cases / total cases Let us examne the results of our system n more detals. The ndexng results of subjects of our system are shon n Fgure 8. For example, the lkeness degrees of TOP 1, TOP 2 and TOP of subject 1 are shon to be 96.45%, 94.64 % and 94.27%. The smlarty beteen these lkeness degrees values suggests that relaton ratos are not enough to prove human perspectve. There s stll a room for argument about ntroducng other face features, such as skn colors and har styles. Let us take another example to evaluate ths system. If e use a ne subject to ndex face mage n a database by our system, hat ll happen? In ths example, e use a photograph captured from one of the authors. fter that, e use t to fnd human faces n the Yale database by our system. The result as shon n Fgure9. The Fgure 9 shos that the author has some features smlar to the TOP sample mages such as der mouth. The TOP smlarty lkeness degrees are above 92%. Suppose the hypothess s correct. Let us attempt to extend the results to lasted places of the result sequence n order correspondng to the Yale Face Database; e call t DOWN. ased on these ponts of ve, DOWN supported that have the lo smlarty lkeness degrees (all bello 80%). Therefore, the method e proposed can apply to match the lkeness degrees facal features. 6. Concluson and Further Works Fe studes have been focus on a human pont of ves to ndex face mage. Ths method has presented a ne knoledge-based face ndexng algorthm, hch s use relatonal ratos of the artsts knoledge.expermental results confrm the theoretcal analyss, the results support that the system provdes less operaton load to the human face mages ndexed on the Internet server. In addton, the system also th the advantage of faster mage ndexng on a database, snce the system just compare the ratonal ratos and dth values of facal features. In ths study, e use the dth/hgh hybrd relaton ratos to be the features of a face for ndexng n future. It s possble that our system be further mproved successful rato th more types of the exstng relaton ratos, such as the relatve rotaton angles, skn colors and har styles. The ssues for addng more feature ratos remans to be further dscussed. Subject TOP 1 TOP 2 TOP 1 s 11=96.45% S 12=94.64% S 1=94.27% 2 S 21=98.58% S 22=95.0% S 2=94.17% S 1=98.2% S 2=96.70% S =96.44% Note: S j means lkeness degrees Fgure 8. Top of the most lkeness mages on the Yale database from subjects by our system.
TOP 1 TOP 2 TOP S 11=94.00% S 12=92.78% S 1=92.72% Don 1 Don 2 Don The author1 S 1D1=74.48% S 1D2=77.59% S 1D=78.82% Fgure 9. The most lkeness and unlkeness face mages on the Yale database hen ndexng by nput face mage not n the database. cknoledgements Ths paper as partally supported by the Natonal Scence Councl n Taan, project number: NSC 94-2520-S-09-001. The authors also ould lke to thank Dr. Kuo-Feng Hang s valuable comments for ths paper. References [1] P. Vola, M. Jones, Robust Real-tme Object Detecton, Techncal Report 2001/01, Compaq CRL, February 2001. [8] H. Chen,Z. Lu, C. Rose, Y. Xu, H.Y. Shum, D. Salesn,"Example-ased Composte Sketchng of Human Portrats", NPR, 2004 [9].S. Georghades, P.N. elhumeur, and D.J. Kregman. From fe to many: Generatve models for recognton under varable pose and lumnaton. In Proc. of the 4th IEEE Internatonal Conference on utomatc Face and Gesture Recognton, 2000 [2] J. hlberg and R. Forchhemer. "Face trackng for modelbased codng and face anmaton", Internatonal Journal on Imagng Systems and Technology. Vol. 1(1), pp. 8-22, 200. [] P. Shh and C. Lu. "Face detecton usng dscrmnatng feature analyss and Support Vector Machne", Pattern Recognton, Vol. 9, Issue 2, Pages 260-276, February 2006. [4] T. F. Cootes, G. J. Edards and C. J. Taylor. "ctve ppearance Models", n Proc. European Conference on Computer Vson 1998 (H.urkhardt &. Neumann Ed.s). Vol. 2, pp. 484-498, Sprnger, 1998. [5] M.. Stegmann and R. Larsen. "Mult-band Modellng of ppearance". Frst Internatonal Workshop on Generatve- Model- ased Vson - GMV. pp. 101-106, DIKU, 2002. [6] F. Y. Shh and C. F. Chuang. utomatc extracton of head and face boundares and facal features. Informaton Scences, Vol. 158, pp. 117-10, January 2004. [7]L. Redman. Ho to Dra Carcatures. Contemporary ooks, 1984.