Recognition continued: discriminative classifiers

Transcription

1 Recognton contnued: dscrmnatve classfers Tues Nov 17 Krsten Grauman UT Austn Announcements A5 out today, due Dec 2 1

2 Prevously Supervsed classfcaton Wndow-based generc object detecton basc ppelne boostng classfers face detecton as case study Hdden Markov Models Revew questons Why s t more effcent to extract Vola-Jones-style rectangular flter responses at multple scales, vs. extract typcal convoluton flter responses at multple scales? What does t mean to be a weak classfer? For a classfer cascade used for object detecton, what propertes do we requre the early vs. later classfers (stages) n the cascade to have? 2

3 Today Sldng wndow object detecton wrap-up Attentonal cascade Pros and cons Object proposals for detecton Supervsed classfcaton contnued Nearest neghbors HMM example Support vector machnes Vola-Jones detector: features Whch subset of these features should we use to determne f a wndow has a face? Consderng all possble flter parameters: poston, scale, and type: 180,000+ possble features assocated wth each 24 x 24 wndow Use AdaBoost both to select the nformatve features and to form the classfer 3

4 Vola-Jones detector: AdaBoost Want to select the sngle rectangle feature and threshold that best separates postve (faces) and negatve (nonfaces) tranng examples, n terms of weghted error. Resultng weak classfer: Outputs of a possble rectangle feature on faces and non-faces. For next round, reweght the examples accordng to errors, choose another flter/threshold combo. Cascadng classfers for detecton Form a cascade wth low false negatve rates early on Apply less accurate but faster classfers frst to mmedately dscard wndows that clearly appear to be negatve 4

5 Tranng the cascade Set target detecton and false postve rates for each stage Keep addng features to the current stage untl ts target rates have been met Need to lower AdaBoost threshold to maxmze detecton (as opposed to mnmzng total classfcaton error) Test on a valdaton set If the overall false postve rate s not low enough, then add another stage Use false postves from current stage as the negatve tranng examples for the next stage Vola-Jones detector: summary Tran cascade of classfers wth AdaBoost Faces New mage Non-faces Selected features, thresholds, and weghts Tran wth 5K postves, 350M negatves Real-tme detector usng 38 layer cascade 6061 features n all layers [Implementaton avalable n OpenCV] 5

6 Vola-Jones detector: summary A semnal approach to real-tme object detecton Tranng s slow, but detecton s very fast Key deas Integral mages for fast feature evaluaton Boostng for feature selecton Attentonal cascade of classfers for fast rejecton of nonface wndows P. Vola and M. Jones. Rapd object detecton usng a boosted cascade of smple features. CVPR P. Vola and M. Jones. Robust real-tme face detecton. IJCV 57(2), Boostng: pros and cons Advantages of boostng Integrates classfcaton wth feature selecton Complexty of tranng s lnear n the number of tranng examples Flexblty n the choce of weak learners, boostng scheme Testng s fast Easy to mplement Dsadvantages Needs many tranng examples Other dscrmnatve models may outperform n practce (SVMs, CNNs, ) especally for many-class problems Slde credt: Lana Lazebnk 6

7 Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng 11/17/2015 Vola-Jones Face Detector: Results Wndow-based detecton: strengths Sldng wndow detecton and global appearance descrptors: Smple detecton protocol to mplement Good feature choces crtcal Past successes for certan classes 7

8 Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng 11/17/2015 Wndow-based detecton: Lmtatons Hgh computatonal complexty For example: 250,000 locatons x 30 orentatons x 4 scales = 30,000,000 evaluatons! If tranng bnary detectors ndependently, means cost ncreases lnearly wth number of classes Wth so many wndows, false postve rate better be low Lmtatons (contnued) Not all objects are box shaped 8

9 Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng 11/17/2015 Lmtatons (contnued) Non-rgd, deformable objects not captured well wth representatons assumng a fxed 2d structure; or must assume fxed vewpont Objects wth less-regular textures not captured well wth holstc appearance-based descrptons Lmtatons (contnued) If consderng wndows n solaton, context s lost Sldng wndow Detector s vew Fgur e cr edt: Der ek Hoem 9

10 Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng 11/17/2015 Lmtatons (contnued) In practce, often entals large, cropped tranng set (expensve) Requrng good match to a global appearance descrpton can lead to senstvty to partal occlusons Image credt: Adam, Rvln, & Shmshon When wll sldng wndow face detecton work best? Class photos 10

11 Vsual Perc eptual Object and Recognton Sensory Augmented Tutoral Computng 11/17/2015 What other categores are amenable to wndowbased representaton? Pedestran detecton Detectng uprght, walkng humans also possble usng sldng wndow s appearance/texture; e.g., SVM wth Haar wav elets [Papag eorgou & Pog go, IJCV 2000] Space-tm e r ectangle featur es [Vola, Jones & Snow, ICCV 2003] SVM wth HoGs [Dalal & Trg gs, CVPR 2005] 11

12 Recap so far Basc ppelne for wndow-based detecton Model/representaton/classfer choce Sldng wndow and classfer scorng Boostng classfers: general dea Vola-Jones face detector Exemplar of basc paradgm Plus key deas: rectangular features, Adaboost for feature selecton, cascade Pros and cons of wndow-based detecton Object proposals Man dea: Learn to generate category-ndependent regons/boxes that have object-lke propertes. Let object detector search over proposals, not exhaustve sldng wndows Alexe et al. Measurng the objectness of mage wndows, PAMI

13 Object proposals Mult-scale salency Color contrast Alexe et al. Measurng the objectness of mage wndows, PAMI 2012 Object proposals Edge densty Superppxel straddlng Alexe et al. Measurng the objectness of mage wndows, PAMI

14 More proposals 11/17/2015 Object proposals Alexe et al. Measurng the objectness of mage wndows, PAMI 2012 Regon-based object proposals J. Carrera and C. Smnchsescu. Cpmc: Automatc object segmentaton usng constraned parametrc mn-cuts. PAMI,

15 Object proposals: Several related formulatons Alexe et al. Measurng the objectness of mage wndows, PAMI 2012 J. Carrera and C. Smnchsescu. Cpmc: Automatc object segmentaton usng constraned parametrc mn-cuts. PAMI, Ian Endres and Derek Hoem. Category-ndependent object proposals wth dverse rankng. In PAMI, Mng-Mng Cheng, Zmng Zhang, Wen-Yan Ln, and Phlp H. S. Torr. BING: Bnarzed normed gradents for objectness estmaton at 300fps. In CVPR, 2014 C. Lawrence Ztnck and Potr Dollár. Edge boxes: Locatng object proposals from edges. In ECCV, J. Ujlngs, K. van de Sande, T. Gevers, and A. Smeulders. Selectve search for object recognton. IJCV, Pablo Arbelaez, J. Pont-Tuset, Jon Barron, F. Marqués, and Jtendra Malk. Multscale combnatoral groupng. In CVPR, Today Sldng wndow object detecton wrap-up Attentonal cascade Pros and cons Object proposals for detecton Supervsed classfcaton contnued Nearest neghbors HMM example Support vector machnes 15

16 Wndow-based models: Three case studes Boostng + face detecton NN + scene Gst classfcaton SVM + person detecton Vola & Jones e.g., Hays & Efros e.g., Dalal & Trggs Nearest Neghbor classfcaton Assgn label of nearest tranng data pont to each test data pont Black = negatve Red = postve from Duda et al. Novel test example Closest to a postve example from the tranng set, so classfy t as postve. Vorono parttonng of feature space for 2-category 2D data 16

17 K-Nearest Neghbors classfcaton For a new pont, fnd the k closest ponts from tranng data Labels of the k ponts vote to classfy Black = negatve Red = postve k = 5 If query lands here, the 5 NN consst of 3 negatves and 2 postves, so we classfy t as negatve. Source: D. Lowe A nearest neghbor recognton example 17

18 Where n the World? [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] Where n the World? 18

19 Where n the World? 6+ mllon geotagged photos by 109,788 photographers Annotated by Flckr users 19

20 6+ mllon geotagged photos by 109,788 photographers Annotated by Flckr users Whch scene propertes are relevant? 20

21 Spatal Envelope Theory of Scene Representaton Olva & Torralba (2001) A scene s a sngle surface that can be represented by global (statstcal) descrptors Slde Credt: Aude Olva Global texture: capturng the Gst of the scene Capture global mage propertes whle keepng some spatal nformaton Olva & Torralba IJCV 2001, Torralba et al. CVPR 2003 Gst descrptor 21

22 Whch scene propertes are relevant? Gst scene descrptor Color Hstograms - L*A*B* 4x14x14 hstograms Texton Hstograms 512 entry, flter bank based Lne Features Hstograms of straght lne stats Im2gps: Scene Matches [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] 22

23 Im2gps: Scene Matches [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] 23

24 [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] Scene Matches [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] 24

25 [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] Quanttatve Evaluaton Test Set 25

26 The Importance of Data [Hays and Efros. m2gps: Estmatng Geographc Informaton from a Sngle Image. CVPR 2008.] Nearest neghbors: pros and cons Pros: Smple to mplement Flexble to feature / dstance choces Naturally handles mult-class cases Can do well n practce wth enough representatve data Cons: Large search problem to fnd nearest neghbors Storage of data Must know we have a meanngful dstance functon Krsten Grauman 26

27 HMM example: Photo Geo-locaton Where was ths pcture taken? Example: Photo Geo-locaton Where was ths pcture taken? 27

28 Example: Photo Geo-locaton Where was ths pcture taken? Example: Photo Geo-locaton Where was each pcture n ths sequence taken? 28

29 Idea: Explot the beaten path Learn dynamcs model from tranng tourst photos Explot tmestamps and sequences for novel test photos [Chen & Grauman CVPR 2011] Idea: Explot the beaten path [Chen & Grauman CVPR 2011] 29

30 Hdden Markov Model P(Observaton State ) Observaton P(State ) P(S2 S2) State 2 P(S3 S2) Observaton P(S2 S1) P(S1 S2) P(S2 S3) Observaton State 1 State 3 P(S1 S1) P(S3 S1) P(S3 S3) P(S1 S3) Dscoverng a cty s locatons Defne states wth data-drven approach: New York mean shft clusterng on the GPS coordnates of the tranng mages 30

31 Observaton model P(Observaton State) = P( Lberty Island) P(L2 L2) P(L2 L1) P(L1 L2) Locaton 2 P(S2 S3) P(L3 L2) P(L1 L1) Locaton 1 P(L3 L1) P(L1 L3) Locaton 3 P(L3 L3) Observaton model 31

32 Locaton estmaton accuracy Qualtatve Result New York 32

33 Dscoverng travel gudes beaten paths Routes from travel gude book for New York vs. Random walks n learned HMM Vdeo textures Schodl, Szelsk, Salesn, Essa; Sggraph

34 Today Sldng wndow object detecton wrap-up Attentonal cascade Pros and cons Object proposals for detecton Supervsed classfcaton contnued Nearest neghbors HMM example Support vector machnes Wndow-based models: Three case studes Boostng + face detecton NN + scene Gst classfcaton SVM + person detecton Vola & Jones e.g., Hays & Efros e.g., Dalal & Trggs 34

35 Lnear classfers Lnear classfers Fnd lnear functon to separate postve and negatve examples x postve : x negatve : x w b 0 x w b 0 Whch lne s best? 35

36 Support Vector Machnes (SVMs) Dscrmnatve classfer based on optmal separatng lne (for 2d case) Maxmze the margn between the postve and negatve tranng examples Support vector machnes Want lne that maxmzes the margn. x postve ( y x negatve ( y 1): 1): x w b 1 x w b 1 x w b For support, vectors, 1 Support vectors Margn C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

37 Support vector machnes Want lne that maxmzes the margn. x postve ( y x negatve ( y 1): 1): x w b 1 x w b 1 Support vectors Margn M For support, vectors, x w b 1 Dstance between pont x w b and lne: w For support vectors: Τ w x b 1 M w w 1 1 w w 2 w Support vector machnes Want lne that maxmzes the margn. x postve ( y x negatve ( y 1): 1): x w b 1 x w b 1 Support vectors Margn M x w b For support, vectors, 1 Dstance between pont x w b and lne: w Therefore, the margn s 2 / w 37

38 Fndng the maxmum margn lne 1. Maxmze margn 2/ w 2. Correctly classfy all tranng data ponts: x postve ( y x negatve ( y 1): 1): x w b 1 x w b 1 Quadratc optmzaton problem: Mnmze 1 2 w T w Subject to y (w x +b) 1 Fndng the maxmum margn lne Soluton: w y x learned weght Support vector 38

39 Fndng the maxmum margn lne Soluton: w y x b = y w x (for any support vector) w x b y x x b Classfcaton functon: f ( x) sgn ( w x b) sgn y x x b If f(x) < 0, classfy as negatve, f f(x) > 0, classfy as postve C. Burges, A Tutoral on Support Vector Machnes f or Pattern Recognton, Data Mnng and Knowledge Dscov ery, CVPR

40 HoG descrptor Dalal & Trggs, CVPR 2005 Person detecton wth HoG s & lnear SVM s Dalal & Trggs, CVPR 2005 Map each grd cell n the nput wndow to a hstogram countng the gradents per orentaton. Tran a lnear SVM usng tranng set of pedestran vs. non-pedestran wndows. 40

41 Person detecton wth HoGs & lnear SVMs Hstograms of Orented Gradents for Human Detecton, Navneet Dalal, Bll Trggs, Internatonal Conference on Computer Vson & Pattern Recognton - June Questons What f the data s not lnearly separable? 41

42 Non-lnear SVMs Datasets that are lnearly separable wth some nose work out great: But what are we gong to do f the dataset s just too hard? 0 x How about mappng data to a hgher-dmensonal space: x 2 0 x 0 x Non-lnear SVMs: feature spaces General dea: the orgnal nput space can be mapped to some hgher-dmensonal feature space where the tranng set s separable: Φ: x φ(x) Slde f rom Andrew Moore s tutoral: m.html 42

43 Nonlnear SVMs The kernel trck: nstead of explctly computng the lftng transformaton φ(x), defne a kernel functon K such that K(x,x j j) = φ(x ) φ(x j ) Ths gves a nonlnear decson boundary n the orgnal feature space: yk ( x, x ) b Kernel trck : Example 2-dmensonal vectors x=[x 1 x 2 ]; let K(x,x j )=(1 + x T x j ) 2 Need to show that K(x,x j )= φ(x ) T φ(x j ): K(x,x j )=(1 + x T x j ) 2, = 1+ x 12 x j x 1 x j1 x 2 x j2 + x 22 x j x 1 x j1 + 2x 2 x j2 = [1 x x 1 x 2 x 2 2 2x 1 2x 2 ] T [1 x j1 2 2 x j1 x j2 x j2 2 2x j1 2x j2 ] = φ(x ) T φ(x j ), where φ(x) = [1 x x 1 x 2 x 2 2 2x 1 2x 2 ] 43

44 Examples of kernel functons Lnear: K( x, x j ) x T x j Gaussan RBF: x x j K( x,x j ) exp( ) Hstogram ntersecton: K ( x, x j ) mn( x ( k), x j ( k)) k SVMs for recognton 1. Defne your representaton for each example. 2. Select a kernel functon. 3. Compute parwse kernel values between labeled examples 4. Use ths kernel matrx to solve for SVM support vectors & weghts. 5. To classfy a new example: compute kernel values between new nput and support vectors, apply weghts, check sgn of output. Krsten Grauman 44

45 Questons What f the data s not lnearly separable? What f we have more than just two categores? Mult-class SVMs Acheve mult-class classfer by combnng a number of bnary classfers One vs. all Tranng: learn an SVM for each class vs. the rest Testng: apply each SVM to test example and assgn to t the class of the SVM that returns the hghest decson value One vs. one Tranng: learn an SVM for each par of classes Testng: each learned SVM votes for a class to assgn to the test example Krsten Grauman 45

46 SVMs: Pros and cons Pros Kernel-based framework s very powerful, flexble Often a sparse set of support vectors compact at test tme Work very well n practce, even wth small tranng sample szes Cons No drect mult-class SVM, must combne two-class SVMs Can be trcky to select best kernel functon for a problem Computaton, memory Durng tranng tme, must compute matrx of kernel values for every par of examples Learnng can take a very long tme for large-scale problems Adapted from Lana Lazebnk Summary Object recognton as classfcaton task Boostng (face detecton ex) Support vector machnes and HOG (person detecton ex) Nearest neghbors and global descrptors (scene rec ex) Sldng wndow search paradgm Pros and cons Speed up wth attentonal cascade Object proposals as alternatve to exhaustve search HMM examples 46