Multclass Object Recognton based on Texture Lnear Genetc Programmng Gustavo Olague 1, Eva Romero 1 Leonardo Trujllo 1, and Br Bhanu 2 1 CICESE, Km. 107 carretera Tjuana-Ensenada, Mexco, olague@ccese.mx, WWW home page: http://cencascomp.ccese.mx/evovson/ 2 Center for Research n Intellgent Systems, Unversty of Calforna, Rversde, USA Abstract. Ths paper presents a lnear genetc programmng approach, that solves smultaneously the regon selecton and feature extracton tasks, that are applcable to common mage recognton problems. The method searches for optmal regons of nterest, usng texture nformaton as ts feature space and classfcaton accuracy as the ftness functon. Texture s analyzed based on the gray level cooccurrence matrx and classfcaton s carred out wth a SVM commttee. Results show effectve performance compared wth prevous results usng a standard mage database. 1 Introducton Recognton s a classcal problem n computer vson whose task s that of determnng whether or not the mage data contans some specfc object, feature, or actvty. Ths task can normally be solved robustly by a human, but s stll not satsfactory solved by a computer for the general case: arbtrary objects n arbtrary stuatons. Genetc and evolutonary algorthms have been used to solve recognton problems n recent years. Tackett [16] presented one of the frst works that appled genetc and evolutonary algorthms to solve recognton problems. The author used genetc programmng (GP) to assgn detected mage features to classfy vehcles such as tanks on US ARMY NVEOD terran board magery. In ths work the genetc programmng approach outperformed a neural network, as well as bnary tree classfer on the same data, producng lower false postves. Teller and Veloso [17, 18] apply also genetc programmng to perform face recognton tasks based on the PADO language usng a local ndexed memory. The method was tested on a classfcaton of 5 classes and acheved 60% of accuracy for mages wthout nose. Howard et al. [7, 8] propose a mult-stage genetc programmng approach to evolve fast and accurate detectors n short evoluton tmes. In a frst stage the GP takes a random selecton of non-object pxels and all the object pxels from the ground truth as test ponts. The fttest detector from the evoluton s appled n order to produce a set of false postves (FP). A second stage of GP uses the dscovered FP and all of the object pxels from the truth as test ponts to evolve a second detector. Then, the fttest detectors
II from both stages are combned and n order to detect objects havng large varablty ths two-stage GP process s further extended nto a number of stages. Ths mult-stage method stops when enough sub-detectors exst to detect all objects. Zhang et al. [20] use GP for doman ndependent object detecton problems n whch the locatons of small objects of multple classes n large mages must be found. They consder three termnal sets based on doman ndependent pxel statstcs and consder also two dfferent functon sets. The ftness functon s based on the detecton rate and the false alarm rate. The approach was tested on three object detecton problems where the objects are approxmately the same sze and the background s not too cluttered. Ln and Bhanu [12] propose a co-evolutonary genetc programmng (CGP) approach to learn composte features for object recognton. The motvaton of usng genetc programmng s to overcome the lmtatons of human experts who consder only a small number of conventonal combnatons of prmtve features. In ths way, ther CGP method can try a very large number of unconventonal combnatons whch may yeld exceptonally good results. Experments wth SAR mages show that CGP could learn good composte features n order to dstngush from several classes. Krawec and Bhanu [10] propose to use lnear genetc programmng to represent feature extracton agents wthn a framework of cooperatve coevoluton n order to learn feature-based recognton tasks. Experments on demandng real-world tasks of object recognton n synthetc aperture radar magery, shows the compettveness of the proposed approach wth human-desgned recognton systems. Roberts and Clardge [15] present a system whereby a feature constructon stage s smultaneously coevolved along sde the GP object detectors. In ths way, the proposed system s able to learn both stages of the vsual process smultaneously. Intal results n artfcal and natural mages show how t can quckly adapt to form general solutons to dffcult scale and rotaton nvarant problems. Ths work proposes a general multclass object recognton system to be tested n a challengng mage database commonly used n computer vson research [2]. Categorzaton s the name n computer vson for the automatc recognton of object classes from mages. Ths task s normally posed as a learnng problem n whch several classes are parttoned nto sets for tranng and testng. The goal s to show that hgh classfcaton accuracy s feasble for three object classes on photographs of real objects vewed under general lghtng condtons, poses and vewponts. The set of test mages used for valdaton comprse photographs obtaned from a standard mage database, as well as mages from the web n order to show the generalty of the proposed approach. The proposed method performs well on texture-rch objects and structure-rch ones, because s based on the cooccurrence matrx. We decde to represent the feature extracton procedure wth ndvduals followng the lnear genetc programmng technque, a hybrd of genetc algorthms and genetc programmng, whch has the advantage of beng able to control the elements n the tree structure. In ths way, each element of the tree structure s evolved only wth the respectve elements of other elements n the populaton. Ths characterstc gves the partcularty of beng postonal allowng the emergence of substructures and avodng the destructve
III effect of crossover, whch s consdered as a mere mutaton n regular GP [1, 13, 10]. 2 Texture Analyss and The Gray Level Cooccurrence Matrx Image texture analyss has been a major research area n the feld of computer vson snce the 1970 s. Hstorcally, the most commonly used methods for descrbng texture nformaton are the statstcal based approaches. Frst order statstcal methods use the probablty dstrbuton of mage ntenstes approxmated by the mage hstogram. Wth such statstcs, t s possble to extract descrptors that characterze mage nformaton. Frst order statstcs descrptors nclude: entropy, kurtoss and energy, to name but a few. Second order statstcal methods represent the jont probablty densty of the ntensty values (gray levels) between two pxels separated by a gven vector V. Ths nformaton s coded usng the Gray Level Cooccurrence Matrx (GLCM) M (, j) [3, 4]. Statstcal nformaton derved from the GLCM has shown relable performance n tasks such as mage classfcaton [5] and content based mage retreval [9, 14]. Formally, the GLCM M,j (π) defnes a jont probablty densty functon f(, j V, π) where and j are the gray levels of two pxels separated by a vector V, and π = {V, R} s the parameter set for M,j (π). The GLCM dentfes how often pxels that defne a vector V (d, θ), and dffer by a certan amount of ntensty value = j appear n a regon R of a gven mage I. Where V defnes the dstance d and orentaton θ between the two pxels. The drecton of V, can or cannot be taken nto account when computng the GLCM. The GLCM presents a problem when the number of dfferent gray levels n regon R ncrease, turnng dffcult to handle or use drectly due to the dmensons of the GLCM. Fortunately, the nformaton encoded n the GLCM can be expressed by a vared set of statstcally relevant numercal descrptors. Ths reduces the dmensonalty of the nformaton that s extracted from the mage usng the GLCM. Extractng each descrptor from an mage effectvely maps the ntensty values of each pxel to a new dmenson. In ths work, the set Ψ of descrptors [4] extracted from M (, j) s formed by the followng: Entropy, Contrast, Homogenety, Local homogenety, Drectvty, Unformty, Moments, Inverse moments, Maxmum probablty, and Correlaton. 3 Evolutonary Learnng of Texture Features The general methodology that s proposed here consders the dentfcaton of Regons of Interests (ROIs) and the selecton of the set of features of nterest (texture descrptors). Thus, vsual learnng s approached wth an evolutonary algorthm that searches for optmal solutons to the multclass object recognton problem. Two tasks are solved smultaneously. The frst task conssts n dentfyng a set of sutable regons where feature extracton s to be performed. The
IV second task conssts n selectng the parameters that defne the GLCM, as well as the set of descrptors that should be computed. The output of these two tasks s taken as nput by a SVM commttee that gves the expermental accuracy on a multclass problem for the selected features and ROIs. Lnear Genetc Programmng for Vsual Learnng Ftness value c w1... c w... c wr 0 x w1 y w1 h w1 w w1 ψ w 1 x w y w h w w w 1 x w y, w 1 0 ψ w 0......... 1 1 0 1 k m t x wr y wr h wr w wr ψ w r.... n h w GLCM w w,...,,...,,..., γ = w, 1 ( β w,1 β w,k β w,m β w,t ) m γ = w, 2 (,...,,..., β β w,m β w,1 β w,k w,t ) m.,...,,...,,..., γ = w, n ( β w,1 β w,k β w,m β w,t ) m Fg. 1. LGP uses a tree structure smlar to the Multcellular Genetc Algorthm [13]. The learnng approach accomplshes a combned search and optmzaton procedure n a sngle step. The LGP searches for the best set Ω of ROIs for all mages and optmzes the feature extracton procedure by tunng the GLCM parameter set π ω Ω through the selecton of the best subset {β 1...β m } of mean descrptor values from the set of all possble descrptors Ψ, to form a feature vector γ = (β 1...β m ) for each ω Ω. Usng ths representaton, we are tghtly couplng the ROI selecton step wth the feature extracton process. In ths way, the LGP s learnng the best overall structure for the recognton system n a sngle closed loop learnng scheme. Our approach elmnates the need of a human desgner, whch normally combnes the ROI selecton and feature extracton steps. Now ths step s left up to the learnng mechansm. Each possble soluton s coded nto a sngle bnary strng. Its graphcal representaton s depcted n fgure 1. The entre chromosome conssts of a tree structure of r bnary and real coded strngs, and each set of varables s evolved wthn ther correspondng group. The chromosome can be better understood by logcally dvdng t n two man sectons. The frst one encodes varables for searchng the ROIs on the mage, and the second s concerned wth settng the GLCM parameters and choosng approprate descrptors for each ROI.
V ROI Selecton The frst part of the chromosome encodes ROI selecton. The LGP has a herarchcal structure that ncludes both control and parametrc varables. The secton of structural or control genes c determne the state (on/off) of the correspondng ROI defnton blocks ω. Each structural gene actvates or deactvates one ROI n the mage. Each ω establshes the poston, sze and dmensons of the correspondng ROI. Each ROI s defned wth four degrees of freedom around a rectangular regon: heght, wdth, and two coordnates ndcatng the central pxel. The choce of rectangular regons s not related n any way wth our vsual learnng algorthm. It s possble to use other types of regons; e.g., ellptcal regons, and keep the same overall structure of the LGP. The complete structure of ths part of the chromosome s coded as follows: 1. r structural varables {c 1...c r }, represented by a sngle bt each. Each one controls the actvaton of one ROI defnton block. These varables control whch ROI wll be used n the feature extracton process. 2. r ROI defnton blocks ω 1...ω r. Each block ω, contans four parametrc varables ω = {x ω, y ω, h ω, w ω }, where the varables defne the ROIs center (x ω, y ω ), heght (h ω ) and wdth (w ω ). In essence each ω establshes the poston and dmenson for a partcular ROI. Feature Extracton The second part of the soluton representaton encodes the feature extracton varables for the vsual learnng algorthm. The frst group s defned by the parameter set π of the GLCM computed at each mage ROI ω Ω. The second group s defned as a strng of eleven decson varables that actvate or deactvate the use of a partcular descrptor β j Ψ for each ROI. Snce each of these parametrc varables are assocated to a partcular ROI, they are also dependent on the state of the structural varables c. They only enter nto effect when ther correspondng ROI s actve (set to 1). The complete structure of ths part of the chromosome s as follows: 1. A parameter set π ω s coded ω Ω, usng three parametrc varables. Each π ω = {R ω, d ω, θ ω } descrbes the sze of the regon R, dstance d and drecton θ parameters of the GLCM computed at each ω. Note that R s a GLCM parameter, not to be confused wth the ROI defnton block ω. 2. Eleven decson varables coded usng a sngle bt to actvate or deactvate a descrptor β j,ω Ψ at a gven ROI. These decson varables determne the sze of the feature vector γ, extracted at each ROI n order to search for the best combnaton of GLCM descrptors. In ths representaton, each β j,ω represents the mean value of the jth descrptor computed at ROI ω. Classfcaton and Ftness Evaluaton Snce the recognton problem ams to classfy every extracted regon ω, we mplement a SVM commttee that uses a votng scheme for classfcaton. The SVM commttee Φ, s formed by the set of all traned SVMs {φ }, one for each ω. The compound feature set Γ = {γ ω } s fed to the SVM commttee Φ, where each γ ω s the nput to a correspondng φ. The SVM Commttee uses votng to determne the class of the correspondng
VI mage. In ths way, the ftness functon s computed wth the Accuracy, whch s the average accuracy of all SVMs n Φ for a gven ndvdual. In other words, Accuracy = 1 Φ x Acc φ x, summed φ x Φ, where Acc φx s the accuracy of the φ j SVM. SVM Tranng Parameters SVM mplementaton was done usng lbsvm [11], a C++ open source lbrary. For every φ Φ, the parameter settng s the same for all the populaton. The SVM parameters are: Kernel Type: A Radal Bass Functon (RBF) kernel was used, gven by: k(x, x ) = exp( x x 2 2σ 2 ) (1) The RBF shows a greater performance rate for classfyng non lnear problems than other types of kernels. Tranng Set: The tranng set used was extracted from 90 dfferent mages. Cross Valdaton: In order to compute the accuracy of each SVM, we perform k-fold cross valdaton, wth k=6. In general, the accuracy computed wth crossvaldaton wll out perform any other type of valdaton approach [6]. In k-fold cross valdaton the data s dvded nto k subsets of (approxmately) equal sze. The SVM was traned k tmes, each tme leavng out one of the subsets from tranng, but usng only the omtted subset to compute the classfers accuracy. Ths process s repeated untl all subsets have been used for both testng and tranng and the computed average accuracy was used as the performance measure for the SVM. 4 Experments wth the CALTECH Image Database The mage database [2] contans 240 mages from whch 120 mages contan several objects and the other 120 correspond to the same mages that have been segmented manually. These mages contan objects wth dfferent lghtng condtons, n dfferent postons, and wth several vewponts. Each mage s recorded n RGB format wth a sze of 320 213 pxels. The objects belong to 7 classes: buldng, trees, cows, arplanes, faces, cars, and bcycles. Because the number of mages was nsuffcent we add more mages from the web. We select three classes to test the proposed system: buldng, faces, and cars. We have two categores for each class: one set of 30 mages for tranng (from [2], see Fgures 2(a), 3(a) y 4(a)) and one set of 50 mages for testng (from the web, see Fgures 2(b), 3(b) y 4(b)). All mages were cropped to gray level wth a sze of 128 128 pxels. The parameters of the LGP were 85% crossover, 15% mutaton, 80 generatons, and 80 ndvduals. Next, two noteworthy ndvdual solutons are presented:
VII (a) Set of tranng mages (b) Set of testng mages Fg. 2. Images for the class buldng. Indvdual 92.22% Ths ndvdual performs very well wth a hgh average accuracy for tranng that acheves 92.22%, whle the testng s qute good wth 73%. Ths dfference s due to the new characterstcs of the mages downloaded from the web. The LGP selects only one bg regon located n the lower part of the mage because most of the cars are n ths part of the mages. The best ndvdual obtaned n ths case s depcted n Fgure 5(a), and a photograph wth the correspondng ROI s shown n Fgure 5(b). Table 4 presents the confuson matrx for ths ndvdual appled on the testng databases. Table 1. Confuson matrx obtaned for the testng set: 73%. Buldng F aces Cars Buldng 68% 20% 12% Faces 18% 78% 4% Cars 14% 14% 72% Indvdual 88.88% Another soluton correspondng to an ndvdual wth an average accuracy for tranng of 88.88% was selected to show the level of classfcaton. Its average durng testng was as hgh as 80% because the set of testng mages s composed only by the more smlar mages wth respect to the tranng stage. Smlar to the prevous case the best ndvdual selects the lower part of the mages due to ts characterstcs. Ths ndvdual s depcted n Fgure 5(c), and a photograph wth the correspondng ROI s shown n Fgure 5(d). Table 4 presents the confuson matrx for ths ndvdual appled on the testng databases.
VIII (a) Set of tranng mages (b) Set of testng mages Fg. 3. Images for the class faces. Table 2. Confuson matrx obtaned for the testng set: 80%. Buldng F aces Cars Buldng 85% 11% 4% Faces 6% 80% 14% Cars 12% 12% 76% Comparson wth Other Approaches The advantage of usng a standard database s that t s possble to compare wth prevous results. For example, n [19] the authors proposed a method that classfes a regon accordng to the proporton of several vsual words. The vsual words and the proporton of each object are learned from a set of tranng mages segmented by hand. Two methods were used to evaluate the classfcaton: nearest neghbor and Gaussan model. On the average [19] acheved 93% of classfcaton accuracy usng the segmented mages; whle on average the same method acheves 76% choosng the regons by hand. Ths last result s comparable to our result. Several aspects could be mentoned: The approach proposed n ths paper does not use segmented mages. The ROI was automatcally selected by the LGP. The mages used n the testng stage does not belong to the orgnal database [2], these mages wth a bgger dfference were obtaned from the web. We could say that the system presents a postve comparson wth respect to the work publshed n [19]. Table 3 shows the comparson of our approach aganst those proposed by [19].
IX (a) Set of tranng mages (b) Set of testng mages Fg. 4. Images for the class cars. Table 3. Comparson of the Recognton Accuracy. NN NN k = 2000 k = 216 Gaussan LGP Feature Selecton Hand Hand Hand Automatc Accuracy 76.3% 78.5% 77.4% 80.0% 5 Conclusons Ths paper has presented a general approach based on lnear genetc programmng to solve multclass object recognton problems. The proposed strategy searches smultaneously the optmal regons and features that better classfy three dfferent object classes. The results presented here show effectve performance compared wth state-of-the-art results publshed n computer vson lterature. Acknowledgments Ths research was funded by a UC MEXUS-CONACyT Collaboratve Research Grant 2005 through the project Intellgent Robots for the Exploraton of Dynamc Envronments. Ths research was also supported by the LAFMI project. Second and thrd authors supported by scholarshps 188966 and 174785 from CONACyT. Frst author gratefully acknowledges the support of Junta de Extremadura granted when Dr. Olague was n sabbatcal leave at the Unversdad de Extremadura n Merda, Span. References 1. Banzhaf, W., Nordc, P., Keller, R., Francne, F.: Genetc programmng: An ntroducton: On the automatc evoluton of computer programs and ts applcatons. San Francsco, CA: Morgan Kaufmann, (1998)
X Frst part: Second part: 1 23 115 123 91 1 1 2 1 1 1 1 1 1 1 1 0 1 0 Best soluton regon 1 x,y h,w V, d, 0 descrptors entropy, contrast, homogenety, local homogenety, correlaton, unformty, 23, 115 123, 91 1, 1, 2 drectvty, frst order dfference moment, nverse moment (a) Best ndvdual wth an Accuracy of 92.22% (b) ROI found by the LGP system Frst part: Second part: 1 56 40 62 123 1 1 3 0 1 1 1 1 0 1 0 0 1 0 Best soluton regon 1 x,y h,w V, d, 0 descrptors 56, 40 62, 123 1, 3, 0 contrast, homogenety, local homogenety, correlaton, drectvty, nverse moment (c) Best ndvdual wth an Accuracy of 88.88% (d) ROI found by the LGP system Fg. 5. Ths mages show the best ndvduals that were found by the LGP approach. 2. CALTECH, Caltech categores, http://www.vson.caltech.edu/html-fles/archve.html, 2005. 3. Haralck, R. M., Shanmugam, K., Dnsten, I.: Textural features for mage classfcaton. IEEE Transactons on Systems, Man and Cybernetcs, 3(6) (1973) 610-621 4. Haralck, R. M.: Statstcal and structural approaches to texture. In proceedngs of the IEEE, 67(5) (1979) 786 804 5. Kjell, J.: Comparatve study of nose-tolerant texture classfcaton. IEEE Internatonal Conference on Systems, Man, and Cybernetcs. Humans, Informaton and Technology, 3 (1994) 2431 2436 6. Goutte, C.: Note on free lunches and cross-valdaton. Neural Computaton, 9(6) (1997) 1245 1249 7. Howard, D., Roberts, S. C., Brankn, R.: Target detecton n SAR magery by genetc programmng. Advances n Engneerng Software. 30 (1993) 303 311 8. Howard, D., Roberts, S. C., Ryan, C.: The boru data crawler for object detecton tasks n machne vson. Applcatons of Evolutonaty Computng, Proceedngs of Evo Workshops 2002. LNCS 2279 (2002) 220 230 9. Howarth, P., Rüger, S. M.: Evaluaton of texture features for content-based mage retreval. Thrd Internatonal Conference on Image and Vdeo Retreval, LNCS 3115 (2004) 326 334 10. Krawec, K., Bhanu, B.: Coevoluton and Lnear Genetc Programmng for Vsual Learnng. In proceedngs of Genetc and Evolutonary Computaton. LNCS 2723 (2003) 332 343 11. Chh-Chung Chang and Chh-Jen Ln. LIBSVM: a lbrary for support vector machnes, 2001. 12. Ln, Y., Bhanu, B.: Learnng features for object recognton. In proceedngs of Genetc and Evolutonary Computaton. LNCS 2724 (2003) 2227 2239 13. Olague, G., Mohr, R.: Optmal camera placement for accurate reconstructon. Pattern Recognton, 34(5) (2002) 927 944 14. Ohanan, P. P., and Dubes, R. C.: Performance evaluaton for four classes of textural features. Pattern Recognton, 25(8) (1992) 819 833 15. Roberts, M. E., Clardge, E.: Cooperatve coevoluton of mage feature constructon and object detecton. Parallel Problem Solvng from Nature. LNCS 3242 (2004) 899 908 16. Tackett, W. A.: Genetc programmng for feature dscovery and mage dscrmnaton. In Stephane Forrest Edtor, Proceedngs of the 5th Internatonal Conference on Genetc Algorthms. Unversty of Illnos at Urbana-Champagn (1993) 303 309 17. Teller, A., Veloso, M.: A controlled experment: Evoluton for learnng dffcult mage classfcaton. In proceedngs of the 7th Portuguese Conference on Artfcal Intellgence. LNAI 990 (1995) 165 176 18. Teller, A., Veloso, M.: PADO: Learnng tree structured algorthms for orchestraton nto an object recognton system. Tech. Rep. CMU-CS-95-101, Department of Computer Scence, Carnege Mellon Unversty, Pttsburgh, Pa, USA, (1995) 19. Wnn, J., Crmns, A., Mnka, T.: Object categorzaton by learned unversal vsual dctonary. 10th IEEE Internatonal Conference on Computer Vson. 2 (2005) 1800 1807 20. Zhang, M., Ceselsk, V., Andreae, P.: A doman ndependent wndow-approach to multclass object detecton usng genetc programmng. EURASIP Journal on Sgnal Processng, Specal Issue on Genetc and Evolutonary Computaton for Sgnal Processng and Image Analyss. 8 (2003) 841 859