A Hierarchical Object Recognition System Based on Multi-scale Principal Curvature Regions

Transcription

1 A Hierarchical Objec Recogniion Sysem Based on Muli-scale Principal Curvaure Regions Wei Zhang, Hongli Deng, Thomas G Dieerich and Eric N Morensen School of Elecrical Engineering and Compuer Science Oregon Sae Universiy, Corvallis, OR 97331, USA {zhangwe deng gd enm}@eecsoregonsaeedu Absrac This paper proposes a new generic objec recogniion sysem based on muli-scale affineinvarian image regions Image segmens are obained by a waershed ransform of he principal curvaure of a conras enhanced image Each region is described by an inensiy-based saisical descripor and a PCA- SIFT descripor The spaial relaions beween regions are represened by a cluser-index disribuion hisogram Wih hese new descripors, we develop a hierarchical objec recogniion sysem which uses an improved boosing feaure selecion mehod [9] o consruc layer classifiers by auomaically selecing he mos discriminaive feaures in each layer All layer classifiers are hen combined o give he final classificaion This sysem is esed on various objec recogniion problems Experimenal resuls show ha he new hierarchical sysem ouperforms he comparable soluions on mos of he daases esed 1 Inroducion The descripion of objec classes is a crucial issue in he design of objec recogniion sysems Previous descripion mehods include single-scale fragmenbased [12] and ineres-region approaches [1,3,5,8,9] While fragmen or par feaures are usually very informaive for objec caegories, hey can be oo classspecific and are no ransform invarian Ineres regions are more generic and more robus o occlusion and ransformaions, bu hey are oo local and ofen noisy Probabilisic consellaion models [5] and clusering-based mehods [3] have been proposed o recognize image caegories based on hese fragmens or ineres regions Insead of describing objecs a a single scale, oher mehods represen objec informaion a muliple scales [2,4,10] These descripions are biologically moivaed he human visual sysem selecs and combines boh coarse (global) and deailed (local) objec feaures for recogniion Shokoufandeh e al [10] use saliency map graphs o capure he salien image srucure using muli-scale wavele ransforms Epshein and Ullman [4] propose feaure hierarchies based on muual informaion feaure selecion and parameer adapaion The work of Bouchard and Triggs [2] model each objec as a hierarchy of pars and subpars wih parial ransformaions (ranslaion and scale ransformaions) ha sofly relae he pars and sub-rees o heir parens Bu here is a common weakness exising in hese hierarchical objec descripions: all hese descripions are highly concree models (rees or graphs) Applying hese ypes of descripions o classificaion requires graph maching [10] or model insaniaion [2,4] algorihms In his paper, we propose a new generic objec descripion which characerizes he global and local feaures of an objec class based on muli-scale principal curvaure regions This new objec descripion mehod is inroduced in Secion 2 Given hese muli-scale descripors, Secion 3 inroduces and deails a hierarchical objec recogniion sysem using an improved boosing feaure selecion mehod Finally, experimenal resuls and conclusions are given in Secions 4 and 5, respecively 2 Muli-scale principal curvaure regions 21 Decoloring To faciliae compuaion of feaure descripors, we conver color images o inensiy images while preserving conras beween regions We employ he decolorizaion algorihm proposed by Grundland and Dodgson [6] o do color conras enhancemen when convering color images o grayscale images This algorihm enhances conras in a meaningful way by adjusing luminance o reflec chromaic differences

2 22 Muli-scale principal curvaure We adap he curvilinear srucures deecor of Seger [11] o generae srucural objec regions defined by he waershed of he image s principal curvaure I has been our experience ha using he principal curvaure produces fairly sable regions ha can be deeced over a range of viewpoins, scales, and appearance changes Furher, hese regions seem more characerisic of objec classes compared wih local corners and blobs The local shape characerisics of an image, viewed as a surface, can be described by he Hessian marix I xx( x,σ D ) I xy ( x,σ D ) H( x, σ D ) = (1) I xy ( x,σ D ) I yy ( x,σ D ) where I xx (x,σ D ), I xy (x,σ D ), and I yy (x,σ D ) are he secondorder derivaives of he image as compued by convolving he image wih he appropriae secondderivaive of a Gaussian wih scale σ D For every pixel, he eigenvalues, (λ 1, λ 2 ), of (1) are proporional o he local principal curvaures while he corresponding eigenvecors, v 1 and v 2, specify he direcions of principal curvaure Each pixel of he principal curvaure image is simply he larges eigenvalue of H(x, σ D ) To reduce he impac of noise, we suppress all principal curvaure pixels (ie, assign hem o zero) ha are below a hreshold τ 0 We hen apply a waershed ransform o he cleaned principal curvaure image using he immersion simulaion mehod proposed by Vincen and Soille [13] Finally, each waershed region is approximaed wih an ellipse having he same second momen To exrac a muli-scale objec descripion ha inegraes boh global and local informaion of objec class, we apply he region deecion algorihm a various scales Figure 1 shows examples of muli-scale principal curvaure regions We see ha a larger scale produces fewer and global regions while a smaller scale resuls in more local feaures 23 Principal curvaure region descripions To more comprehensively describe each region, we employ boh saisical measuremens of region inensiies and PCA-SIFT [7] feaures The saisical feaure combines coefficien of variaion, skewness, kurosis, and momen invarians o form a 9- dimensional feaure vecor for each region The PCA- SIFT feaures are 36-dimensional and have been demonsraed o be more compac and disincive han SIFT [7] (a) Original image (b) Principal curvaure image a scale = 40 (c) Segmened regions a scale = 40 (d) Deeced feaures a scale = 40 (e) Deeced feaures a scale = 20 (f) Deeced feaures a scale = 10 Figure 1 Muli-scale region deecions

3 Global Local Layer classifier L 1: scale s 1 Layer classifier L 2: scale s 2 Layer classifier L n: scale s n Figure 2 Hierarchical objec recogniion sysem In addiion, we characerize he spaial configuraion of he regions wih bins-based cluser index disribuion hisograms The consrucion of spaial relaion feaures involves hree seps Firs, we cluser he PCA-SIFT feaures from he posiive raining images using E-M o fi a Gaussian mixure model wih C = 16 clusers Second, for each region in he raining and esing images, we compue he index of he Gaussian cluser mos likely o have generaed is PCA-SIFT vecor And hird, we discreize he disances and direcions beween regions ino M = 36 bins wih 12 direcions and 3 disance ranges The sizes of he bins are fixed relaive o he image sizes Thus, he spaial configuraion of regions in each image is described by a hisogram R composed of D = C M C = = 9216 feaure elemens An elemen R m,j in R records he number of imes a region wih cluser index j falls ino bin m wih cener region index i 3 Hierarchical objec recogniion sysem Using our new objec descripions, we designed a hierarchical objec recogniion sysem which uses muli-scale image analysis o do classificaion This sysem is illusraed in Figure 2 From he op layer o he boom, we rain layer classifiers L 1,,L n based on he region feaures obained a scales s 1,,s n, which are in decreasing order (global o local) We hen combine he oupus of layer classifiers o predic he class labels of new images + Class labels 31 Layer classifier Using our new descripion mehod above, objec images are described by normal feaure vecors of hree ypes (inensiy saisical feaures, PCA-SIFT, and spaial relaion feaures) insead of concree models This permis sandard classificaion algorihms o be employed as layer classifiers According o our experimens, we noiced ha for mos of he image ses, only a small porion of he image feaures are useful for classificaion So we employ and improve he boosing feaure selecion algorihm proposed by Opel e al [9] ha searches among all he available feaures and auomaically selecs he mos sable and discriminaive ones o form he final classifier The layer classifiers are learned using he AdaBoos algorihm which mainains a weigh for each raining image In ieraion of AdaBoos, all he unseleced feaure vecors of he raining images are evaluaed based on he curren image weighs o find he mos discriminaive feaure We evaluae he saisical inensiy feaures and he PCA-SIFT feaures in he same way as Opel e al [9] The sabiliy and discriminaing power of a feaure vecor v f is evaluaed in hree seps Firs, calculae he disance from v f o each of he raining images This is done by finding he minimum disance beween v f and all he feaure vecors of he same ype in he raining image We use he Mahalanobis disance meric for he saisical inensiy feaure and he Euclidean disance for PCA-SIFT Second, sor he raining images ino ascending order according o heir disances o v f Third, we apply he scanline algorihm [9] o he sored disance array o deermine a hreshold θ f ha maximizes he weighed accuracy of using v f as a weak classifier The maximal weighed sum is adoped as he evaluaion of v f Evaluaing he spaial relaion feaures is simpler because here is no need o calculae he feaure-oimage disances The raining images are direcly sored according o heir spaial relaion feaure values More specifically, all he spaial relaion feaures of K raining images are assembled ino a D K marix A (where D is he dimension of he spaial configuraion hisogram) Then for each row of A, raining images are sored by decreasing order of heir corresponding feaure values Finally, he scanline algorihm scans he sored array and oupus he opimal hreshold and he maximal weighed sum evaluaion for he row, which indicaes he significance of he specific spaial configuraion for classificaion A perfec feaure should have all of he posiive images (+1) sored before all he negaive images ( 1) so ha he feaure vecor gives a weak classifier ha is perfecly discriminaive The feaure and hreshold {v*,

4 θ*} which has maximal score among all he available feaure vecors is seleced as he weak classifier for ieraion We consruc T weak classifiers for each layer All hese T weak classifiers are hen combined ino a srong classifier (called he layer classifier) using sandard AdaBoos The oupu of a srong classifier L i is given by Half of he images in each se are used for raining, and he res are held ou for esing Recogniion performance is evaluaed by ROC equal error raes wih y i = T = 1 (ln β (2) ) h ( I) β 1 ε = (3) ε where h (I) represens he oupu of he h weak classifier of layer classifier L i ε is he weighed classificaion error rae of he h weak classifier compued based on he AdaBoos weighs For presence/absence 2-class objec recogniion problems, i is no plausible o use background feaures o recognize objec examples So we modified he original algorihm in [9] o selec only among he feaures from posiive images 32 Final classificaion The final resul of he hierarchical sysem is simply he sign of he sum of he oupus of layer classifiers, which is given by: n y i i= 1 Y = sign( ) (4) In our enaive experimens, we also ried o se weighs for layer classifiers, and use he Voed Percepron algorihm o adap he weighs o minimize he classificaion error on raining images, bu i overfis he daa and he performance degrades 4 Experimenal resuls We did experimens on various 2-class objec recogniion image ses in order o es he performance of our sysem We employed a four-layer sysem wih scales {40, 30, 20, 10} and he number of boosing ieraions T = 100 The sysem is esed on six objec classes in he Calech daase 1 : airplanes (1074), cars (rear) (526), cars (side) (123), faces (450), leaves (186) and moorbikes (826) The background se in Calech conains 451 images We also esed on a sonefly larva se conaining 70 Doroneuria images (posiive) and 57 Hesperoperla images (negaive) Examples of Calech images and sonefly images are shown in Figure 3 1 hp://wwwvisioncalechedu/feifeili/daaseshm Figure 3 Sample images from Calech and sonefly larva daase wih rows corresponding o: airplanes, cars (rear), cars (side), faces, leaves, moorbikes, Calech background, and Doroneuria (lef wo images) and Hesperoperla (righ wo images) The hierarchical sysem based on he new descripions is esed on hese daases and compared wih he consellaion model of Fergus e al [5] and he boosing feaure selecion approach by Opel e al [9] The resuls are summarized in Table 1 The comparison indicaes ha our hierarchical objec recogniion sysem ouperforms he oher mehods on mos of he comparable daases In order o es he value of our hierarchical srucure, we compared he equal error raes of he whole 4-layer sysem (denoed as 4-layer wih spaial) o he bes single layer classifier (1-layer) The resuls are summarized in he second and hird columns of Table 2 In he forh column of Table 2, we show he performance of he 4-layer sysem wihou spaial relaion (4-layer wihou spaial) o es he uiliy of he spaial configuraion descripor We noiced ha on all hese daases, here are significan gaps beween he performance of he muli-

5 layer sysem and ha of he bes one-layer classifier This demonsraes ha he muli-scale objec descripion is more generic and informaive for objec classes han single scale descripion On mos of he daases, spaial relaion feaures improve he performance of he sysem, hus supporing our claim ha spaial configuraions of deeced regions are also valuable cues for recogniion Table 1 ROC equal error raes of our approach and oher approaches Daase Ours Fergus [5] Opel [9] Airplane Cars(rear) / Cars(side) Faces Leaves 975 / / Moorbikes Soneflies 886 / / Table 2 ROC equal error raes of he 4-layer classifier wih spaial relaion feaures compared o 1-layer classifier and 4-layer classifier wihou spaial relaion feaures Daase 4-layer wih spaial 1-layer 4-layer wihou spaial Airplanes Cars(rear) Cars(side) Faces Leaves Moorbikes Soneflies Conclusion and fuure work In his paper, we propose a novel objec descripion based on muli-scale principal curvaure regions This descripion is invarian o roaion and view ransformaions and robus o scale changes The exure and geomeric informaion of deeced regions are represened by heir inensiy-based and spaial relaion feaures respecively A generic hierarchical objec recogniion sysem using boosing feaure selecion is developed and ouperforms wo oher approaches on various objec classes There are wo fuure direcions we wish o invesigae One is o furher improve he robusness of he principal curvaure region deecor The oher is incorporaing iner-layer spaial relaions ino he hierarchical sysem o furher exploi he spaial consrains 6 Acknowledgemens This work is suppored by Naional Science Foundaion gran number eniled "ITR: Paern Recogniion for Ecological Science and Environmenal Monioring" We would like o hank Calech Vision Lab and Yan Ke for providing es daase and source code for descripor 7 References [1] S Agarwal, A Awan and D Roh, Learning o Deec Objecs in Images via a Sparse, Par-Based Represenaion, IEEE PAMI, 26(11): , 2004 [2] G Bouchard and Bill Triggs, Hierarchical par-based visual objec caegorizaion, CVPR, 2005 [3] G Dorko and C Schmid, Objec Class Recogniion Using Discriminaive Local Feaures, submied o PAMI, 2004 [4] B Epshein and S Ullman Feaure Hierarchies for Objec Classificaion, ICCV, 2005 [5] R Fergus, P Perona, and A Zisserman, Objec Class Recogniion by Unsupervised Scale-Invarian Learning, CVPR, 2003 [6] M Grundland and N A Dodgson, The Decolorize Algorihm for Conras Enhancing, Color o Grayscale Conversion, Technical Repor UCAM-CL-TR-649, Universiy of Cambridge [7] Y Ke and R Sukhankar, PCA-SIFT: A More Disincive Represenaion for Local Image Descripors, CVPR, 2004 [8] J Maas, O Chum, M Urban, T Pajdla, Robus Wide Baseline Sereo from Maximally Sable Exremal Regions, BMVC, 2002 [9] A Opel, M Fussenegger, A Pinz, and P Auer Weak Hypoheses and Boosing for Generic Objec Deecion and Recogniion, ECCV, 2004 [10] A Shokoufandeh, I Marsic, and S J Dickinson, View-Based Objec Recogniion Using Saliency Maps, Image and Vision Compuing, 17(5 6): (1999)8 [11] C Seger, An Unbiased Deecor of Curvilinear Srucures, in IEEE PAMI, 20(2): , 1998 [12] S Ullman, E Sali and M Vidal-Naque, A Fragmen- Based Approach o Objec Represenaion and Classificaion, in Inernaional Workshop on Visual Form, Berlin: Springer, , 2001 [13] L Vincen and P Soille, "Waersheds in Digial Spaces: An Efficien Algorihm Based on Immersion Simulaions," IEEE PAMI, 13(6): , 1991