Learning Statistical Structure for Object Detection

Transcription

1 To appear n AI 003 Learnng tatstcal tructure for Obect Detecton Henry chnederman Robotcs Insttute arnege Mellon Unversty ttsburgh A 53 UA hws@cs.cmu.edu Abstract. Many classes of mages ehbt sparse structurng of statstcal dependency. Each varable has strong statstcal dependency wth a small number of other varables and neglgble dependency wth the remanng ones. uch structurng maes t possble to construct a powerful classfer by only representng the stronger dependences among the varables. In partcular a semnaïve Bayes classfer compactly represents sparseness. A sem-naïve Bayes classfer decomposes the nput varables nto subsets and represents statstcal dependency wthn each subset whle treatng the subsets as statstcally ndependent. However learnng the structure of a sem-naïve Bayes classfer s nown to be N complete. The hgh dmensonalty of mages maes statstcal structure learnng especally challengng. Ths paper descrbes an algorthm that searches for the structure of a sem-naïve Bayes classfer n ths large space of possble structures. The algorthm sees to optmze two cost functons: a localzed error n the log-lelhood rato functon to restrct the structure and a global classfcaton error to choose the fnal structure. We use ths approach to tran detectors for several obects ncludng faces eyes ears telephones push-carts and door-handles. These detectors perform robustly wth a hgh detecton rate and low false alarm rate n unconstraned settngs over a wde range of varaton n bacground scenery and lghtng. Introducton Many classes of mages have sparse structurng of statstcal dependency. Each varable has strong statstcal dependency wth a small number of other varables and neglgble dependency wth the remanng ones. For eample geometrcally algned mages of faces cars push-carts and telephones ehbt ths property. Fgure shows emprcal mutual nformaton among wavelet varables representng frontal human face mages. Mutual nformaton measures the strength of the statstcal dependence between two varables. Each mage s a vsualzaton of the mutual nformaton values between one chosen wavelet varable ndcated by an arrow and all the other varables n the wavelet transform. The brghtness at each locaton ndcates the mutual nformaton between the varable at ths locaton and the chosen varable. These eamples llustrate common behavor where a gven varable s statstcally related wth a small number of other varables.

2 hosen varable hosen varable hosen varable Fg.. Emprcal mutual nformaton among wavelet varables sampled from frontal faces mages. parse structurng of statstcal dependency eplans the emprcal success of parts-based methods for face detecton and face recognton [][][3][4][5]. uch parts-based methods concentrate modelng power on localzed regons n whch dependences tend to be strong and use weaer models over larger areas n whch dependences tend to be less sgnfcant. In general though the problem of actually learnng the statstcal dependency structure for mage classfcaton has not receved much attenton n the computer vson communty. revous parts-based methods use hand-pced parts. In ths paper we propose a method to learn the dependency structure from data and use the dependency structure to buld a sem-naïve Bayes classfer. A sem-naïve Bayes classfer [6] decomposes the nput varables nto subsets representng statstcal dependency wthn each subset whle treatng the subsets as statstcally ndependent. Ths classfer f... n taes the followng form when wrtten as a log-lelhood rato test: f... n r = log + log log r > λ... r {... n} where... n are the nput varables... r are subsets of these varables and and ndcate the two classes. For the problem of obect detecton the classes are obect and non-obect where the non-obect class represents all possble vsual scenery that does not contan the obect. For eample may correspond to face and may correspond to non-face. In ths form the classfer chooses class f f... n > λ. Otherwse t chooses class. Learnng the structure of a sem-naïve Bayes classfer s challengng. The search space s enormous. It s super-eponental n n nput varables where n typcally s ~0 3 for mage classfcaton. Moreover the soluton s N complete; that s we must compare every possble structure n order to fnd the optmal soluton. The Bayesan score [9][4][5] s an deal metrc for comparng these model structures. It naturally penalzes for overfttng. However computng the score for one model nvolves summng the probabltes of the tranng data over all possble nstantatons of the model s parameters. The computatonal cost of dong so can be qute large.

3 oluton usng heurstc search and appromate metrcs s unavodable. On lower dmensonal domans e.g. under a hundred varables proposed methods have focused on onng one varable at a tme usng estmates of par-wse dstrbutons [6] or accuracy usng cross-valdaton [7]. Another method [8] nduces products of decson tree le structures. Our strategy selects a structure by sequentally optmzng two cost functons usng greedy search technques. The frst functon models local error n the log lelhood rato functon over pars of varables. Ths functon assumes every par of varables s ndependent from the remanng varables. We organze the varables nto subsets such that ths measure s mnmzed. In partcular we generate a large sem-redundant pool of canddate subsets. The second optmzaton chooses the fnal soluton as a subset of these canddate subsets by mnmzng a global measure of classfcaton error. We use ths method n the contet of obect detecton. Obect detecton s the tas of fndng nstances of the gven obect anywhere n an mage and at any sze. To perform detecton we use a classfer that dscrmnates between the obect and scenery that does not contan the obect. Ths classfer operates on fed sze nput wndow e.g. 34 for frontal faces and allows for a lmted amount of varaton n sze and algnment of the obect wthn ths wndow. Therefore to perform detecton we ehaustvely scan the classfer over the nput mage e.g. wth a step sze of 4 pels and reszed versons of the nput mage e.g. wth a scale factor step sze of.89. We use ths approach to construct classfers for detectng several types of obects: faces eyes ears telephones push-carts and door-handles. These detectors perform robustly wth a hgh detecton rate and low false alarm rate n unconstraned scenery over a wde range of varaton n bacground scenery and lghtng. onstructon of the lassfer There are two aspects to constructng the classfer: learnng the structure of the classfer assgnment of the varables to subsets n equaton and estmatng probablty dstrbutons over each such subset. In our approach these two steps are coupled. ecton. descrbes the ntal selecton of a pool of canddate subsets. ecton. descrbes the estmaton of probablty dstrbutons over these canddate subsets. ecton.3 descrbes the selecton of a subset of the canddate subsets to form the fnal classfer. The tranng data conssts of mages of the obect for class and varous nonobect mages for class. The nput to the classfer... n s a wavelet transform of nput wndow. However there s nothng about ths method that s specfc to the wavelet transform. onceavably the raw pel varables or any transform of the mage could be used. For more detals about our mage pre-processng tranng data and overall system for detecton refer to [].

4 . Mnmzng Local Error n the Log-Lelhood Rato Ths step creates a large collecton of subsets of varables. uch a selecton of subsets reduces representatonal power. Only dependences wthn each subset are represented. We therefore must decde whch varables we wll not represent and whch dependences we wll not represent. We evaluate the cost of a proposed reducton by ts error n modelng the log-lelhood rato functon. Our error metrc s the dfference between the true log-lelhood rato functon and the log-lelhood rato under the gven reducton. However we compute ths error only over pars of varables. In partcular we consder three possble cases gven by the followng costs: ] log [log abs = ] log [log abs = 3 ] [log 3 = abs 4 Note each of the random varables s assumed to be dscrete-valued. We use upper case notaton to denote the random varable and lower case notaton to denote a partcular nstantaton of the varable; that s each sum s over all possble values of the gven random varable. s the error n modelng the two varables and as ndependent; that s the cost of removng the dependency between the two varables. s the error of removng one varable from the par. 3 s the error of removng both varables from the par. Each of these assumes that the par s ndependent from the remanng nput varables. We obtan these measures by emprcally estmatng the probablty dstrbutons and for every parngs of varables. Under these appromatons the error assocated wth a gven choce of subsets F = {... r } can be computed as: 3 F E + + = I 5 We see a set of canddate subsets F to mnmze ths localzed error functon. We search for such a soluton usng two steps. The frst step assgns the varables to n subsets usng n greedy searches where each nput varable s a seed for one search. Ths guarantees that every varable s represented n at least one subset and therefore there are no errors of the form or 3 for ths step. Ths s a farly reasonable way to optmze EF snce the errors due to removng a varable tend to be greater than those of removng a dependency. Each of the greedy searches adds new varables by choosng the one that has the largest sum of values formed by ts parng wth all

5 current members of the subset. uch a selecton process wll guarantee that the varables wthn any subset wll have strong statstcal dependency wth each other. A second search may be desrable to reduce the number of subsets to smaller collecton. We propose sequentally removng subsets untl some desrable number q are remanng. At each step we remove the subset that wll lead to the smallest ncrease n modelng error. In partcular t follows from equaton 5 that the error n removng a gven subset s: I In the eperments we descrbe later the number of selected canddate subsets q ranged from 00 to 000. However computatonal cost s lnear n the number of canddate subsets and s not prohbtve for large numbers. The szes of the subsets are somewhat of an open queston. Larger subsets have the potental to capture greater dependency. ubset sze however ncreases the dmenson of the probablty dstrbutons n equaton and therefore sze must be balanced aganst practcal lmts n representatonal power and lmted tranng data. One possble way of addressng ths ssue s terms of V dmenson as descrbed by [4] or by Bayesan scorng technques [9][4][5] For smplcty we descrbe the algorthm assumng the subsets wll all have the same number of members however n our eperments we consder multple szes leadng to ntally mn subsets above where m s the number of szes to allow for greater varety n the representaton... Estmatng the robablty Dstrbutons Ths step estmates log-lelhood rato functons log / for each canddate subset. Any functonal form e.g. Gaussan mture model Bayes net non-parametrc etc. s admssble as a choce for and. In general classfcaton functons such as lnear and quadratc dscrmnants neural networs or decson trees may also be admssble for log / by proper normalzaton. In our current eperments we represent each probablty dstrbuton by a table. Ths representaton dscretzes each subset of wavelet varables to a dscrete feature value by vector quantzaton. ee [ for more detals on ths representaton. The probablty tables are then estmated by countng the frequency of occurrence of each feature value n the tranng data..3. Mnmzng Global lassfcaton Error We now form the overall structure of the sem-naïve Bayes classfer by choosng a group of the canddate subsets to form the fnal classfer. We choose the combnaton that mnmzes an emprcal classfcaton error score. We measure performance by the area under the recever operatng characterstc RO [3]. Ths measure of classfcaton error accounts for the classfer s full operatng range over values for the threshold λ n equaton.

6 The dffculty n mang ths selecton s that combnatoral space of canddate subsets s enormous. We use greedy search to ncrementally combne subsets. In ths search the cost of evaluatng each canddate combnaton on an eample s small. In partcular the evaluaton over any combnaton of subsets taes the form of equaton and s therefore smply the sum of the evaluatons over the ndvdual canddate subsets. Therefore the ndvdual canddate log-lelhood rato functons only have to be evaluated once on each eample. We then evaluate any combnaton as a sum of the approprate pre-computed values. In practce t may be desrable to repeat ths process several tmes where each tme we prohbt dentcal choces to the prevous searches. In partcular n our eperments we use a two part strategy that frst fnds l canddate combnatons by comparng performance on tranng data same mages used to estmate probablty dstrbutons then chooses the best of these by comparng performance on cross-valdaton data mages that are separate from other aspects of tranng. 3. Obect Detecton Eperments We used ths method to tran detectors for frontal faces eyes ears telephones pushcarts and door-handles. In frontal face detecton ths method acheves relatvely accurate detecton rates at a farly low computatonal cost. The table below show results on the MIT-MU test set [0][]. Recognton rate 86.5% 90.9% 94.0% 96.% False detectons These results are at least equal and perhaps superor to those of other state of the art detectors on ths testng set ncludng [][][4][5][0][]. A human eye detector traned by ths method has been tested etensvely. The eyes were successfully located wthn a radus of 5 pels wth an accuracy of 98.% on over 9000 mages of faces n an eperment ndependently conducted at the Natonal Insttute of tandards and Technology NIT by NIT employees and reported bac to the author. Ths dataset s sequestered and s not avalable to the publc. The dataset conssts of mugshot stll mages where there s only one face per person n the mage and the face s that most promnent obect n the mage. The algorthm assumed that one face was present per mage for ths eperment. The telephone detector was tested on one model of telephone over a set of 43 mages wth 07 telephones. The telephones had small varatons n desgn colorng and age etc. ome eamples are shown n Fgure 5. The table below gves the performance over dfferent values of the classfcaton threshold n each column: Recognton rate 6.7% 78.5% 85.5% 9.6% False Detectons The author would le to acnowledge Jonathon hllps atrc Grother and am Trahan for ther assstance n runnng these eperments.

7 Accurate and effcent detectors were also traned for human ears push-carts and door-handles and are llustrated n Fgures 5 where graphc overlays ndcate the detected postons of these obects. 4. oncluson parse structurng of statstcal dependency maes t possble to construct a powerful classfer by only representng the stronger dependences among a group of varables. We have llustrated how such structure can be eploted by a sem-naïve Bayes classfer model. In partcular we have shown that the structure of the classfer can be learned by a search that optmzes a local error crteron gven by equaton 5 followed by another search that optmzes a global error crteron descrbed n ecton.3. We have shown that such a classfer can be effectve for dffcult obect detecton tass. We beleve these technques of learnng statstcal structure wll carry over to more comple models. In partcular a sem-naïve Bayes model s the most basc form of the larger graphcal probablty famly of models ncludng Bayes nets Marov random felds factor graphs chan graphs and mtures of trees. uch models mae t possble to represent more comple structural relatonshps such as condtonal ndependence and hold further promse for mproved mage classfcaton and obect recognton. References. chnederman H. Kanade T. Obect Detecton usng the tatstcs of arts. To appear n Internatonal Journal of omputer Vson Rowley H.A. Balua. and Kanade T. Neural Networ-Based Face Detecton. IEEE Transactons on attern Analyss and Machne Intellgence 0: Moghaddam B.; entland A. robablstc vsual learnng for obect representaton. IEEE Transactons on attern Analyss and Machne Intellgence 9: Hesele B. T. erre M. ontl and T. oggo. omponent Based Face Detecton. VR Vola. and Jones M. Rapd Obect Detecton Usng a Boosted ascade of mple Features. VR Kononeno I. em-naïve Bayesan lassfer. th European Worng esson on Learnng. pp Domngos. azzan M.. On the Optmalty of the mple Bayesan lassfer under Zero-One Loss. Machne Learnng. 9: Roach L. and Mamon O. Theory and Applcatons of Attrbute Decomposton. IEEE Internatonal onference on Data Mnng. pp ooper G. and Hersovts E. A Bayesan Method for the Inducton of robablstc Networs from Data. Machne Learnng. 9: ung K-K. oggo T.. Eample-Based Learnng for Vew-Based Human Face Detecton. IEEE Transactons on attern Analyss and Machne Intellgence 0: Roth D. Yang M-H. Ahua N. A NoW-Based Face Detector. N chnederman H. MU Robotcs Insttute Tech Report. In reparaton.

8 3. Duda R. O. Hart. E. tor D. G. attern lassfcaton. John Wley & ons Hecerman D. Geger D. hcerng D. H. 995 Learnng Bayesan Networs: The ombnaton of Knowledge and tatstcal Data. Machne Learnng 03: Fredman N. and Koller D. 00 Beng Bayesan about Networ tructure: A Bayesan Approach to tructure Dscovery n Bayesan Networs. Machne Learnng Journal. Fg.. Face eye and ear detecton Fg. 3. Door-handle detecton Fg. 4. Telephone detecton Fg. 5. ush-art detecton