CMPSCI 670: Computer Vision! Object detection continued. University of Massachusetts, Amherst November 10, 2014 Instructor: Subhransu Maji

Transcription

1 CMPSCI 670: Computer Vson! Object detecton contnued Unversty of Massachusetts, Amherst November 10, 2014 Instructor: Subhransu Maj

2 No class on Wednesday Admnstrva Followng Tuesday s schedule ths Wednesday Offce hours ths week are at Thursday 3:45-4:45 pm 2

3 Object detecton Speedng up t up Makng t more accurate Lecture overvew Today s lecture Recap last lecture (HOG, template matchng, tranng) Issues wth the sldng wndow detector Selectve search usng regon proposals Fast kernel SVM classfers 3

4 Detecton = repeated classfcaton face or not? Detecton 4

5 Hstograms of orented gradents (HOG) Introduce nvarance Bas / gan / nonlnear transformatons - bas: gradents / gan: local normalzaton - nonlnearty: clampng magntude, orentatons Small deformatons! - spatal subsamplng - local bag models References Hstograms of orented gradents for human detecton. N. Dalal and B. Trggs, CVPR Fndng people n mages and vdeos. N. Dalal, Ph.D. Thess, Insttut Natonal Polytechnque de Grenoble,

6 Hstograms of orented gradents (HOG) Partton mage nto blocks at multple scales and compute hstogram of gradent orentatons n each block 10x10 cells 20x20 cells N. Dalal and B. Trggs, Hstograms of Orented Gradents for Human Detecton, CVPR 2005 Image credt: N. Snavely 6

7 Template matchng wth HOG HOG feature map Template Detector response map Compute the HOG feature map for the mage Convolve the template wth the feature map to get score Fnd peaks of the response map (non-max suppresson) What about mult-scale? 7

8 Mult-scale template matchng p (, ) =w (, ) (f) Image pyramd HOG feature pyramd Compute HOG of the whole mage at multple resolutons Score each sub-wndows of the feature pyramd

9 Example pedestran detectons [Dalal06] 9

10 Mnng hard negatves Pos ={......} probabltes to be dstngushed more easly. We wll often Negrand = {... random use mss rate at 10background patches...} 4 FPPW as a reference pont for results. SVM Fgure 2. Some sample mages from our new human detecton database. The subjects are always uprght, but wth some partal occlusons and a wde range of varatons n pose, appearance, clothng, llumnaton and background. Ths s arbtrary but no more so than, e.g. Area Under ROC. In a multscale detector t corresponds to a raw error rate of about0.8 false postvesper mage tested. (The full detector has an even lower false postve rate owng to nonmaxmum suppresson). Our DET curves are usually qute shallow so even very small mprovements n mss rate are equvalent to large gans n FPPW at constant mss rate. For example, for our default detector at 1e-4 FPPW, every 1% absolute (9% relatve) reducton n mss rate s equvalent to reducng the FPPW at constant mss rate by a factor of Overvew of Results Before presentng our detaled mplementaton and performance analyss, we compare the overall performance of our fnal HOG detectors wth that of some other exstng methods. Detectors based on rectangular (R-HOG) or crcular log-polar (C-HOG) blocks and lnear or kernel SVM are compared wth our mplementatons of the Haar wavelet, PCA-SIFT, and shape context approaches. Brefly, these approaches are as follows: Hard negatves Generalzed Haar Wavelets. Ths s an extended set of orented Haar-lke wavelets smlar to (but better than) that used n [17]. The features are rectfed responses from 9 9 and orented 1 + Neghard = {... wndows wth score >= -1...} st and 2 nd dervatve box flters at 45 ntervals and the correspondng 2 nd dervatve xy flter. PCA-SIFT. These descrptors are based on projectng gradent mages onto a bass learned from tranng mages usng PCA [11]. Ke & Sukthankar found that they outperformed SIFT for key pont based matchng, but ths s controversal [14]. Our mplementaton uses blocks wth the same dervatve scale, overlap, etc., settngs as our HOG descrpstrength and edge-presence based votng were tested, wth the edge threshold chosen automatcally to maxmze detecton performance (the values selected were somewhat varable, n the regon of graylevels). Results. Fg. 3 shows the performance of the varous detectors on the MIT and INRIA data sets. The HOG-based detectors greatlyoutperformthe wavelet, PCA-SIFT and Shape Context ones, gvng near-perfect separaton on the MIT test set and at least an order of magntude reducton n FPPW on the INRIA one. Our Haar-lke wavelets outperform MIT wavelets because we also use 2 nd order dervatves and contrast normalze the output vector. Fg. 3(a) also shows MIT s best parts based and monolthc detectors (the ponts are nterpolated from [17]), howeverbewarethat an exact comparson s not possble as we do not know how the database n [17] was dvded nto tranng and test parts and the negatve mages used are not avalable. The performances of the fnal rectangular (R-HOG) and crcular (C-HOG) detectors are very smlar, wth C-HOG havng the slght edge. Augmentng R-HOG wth prmtve bar detectors (orented 2 nd dervatves R2-HOG ) doubles the feature dmenson but Descrptor Cues further mproves the performance (by 2% at 10 4 FPPW). Replacng the lnear SVM wth a Gaussan kernel one mproves performance by about 3% at 10 4 FPPW, at the cost of much hgher run tmes 1. Usng bnary edge votng (EC- HOG) nstead of gradent magntude weghted votng (C- HOG) decreases performance by 5% at 10 4 FPPW, whle omttng orentaton nformaton decreases t by much more, even f addtonal spatal or radal bns are added (by 33% at 10 4 FPPW, for both edges (E-ShapeC) and gradents (G- ShapeC)). PCA-SIFT also performs poorly. One reason s that, n comparson to [11], many more (80 of 512) prncpal vectors have to be retaned to capture the same proporton of SVM 10

11 Poselets for person Bourdev et al

12 Person detecton usng poselets Detect each poselet n an mage Vote for the person boundng box Fnd non-overlappng clusters Score each cluster usng a weghted combnaton of poselet detecton scores s = X p2c w p a p person detecton score weght of each poselet poselet detecton score Bourdev & Malk 09, Bourdev et al.10, Maj & Malk 10 12

13 Issues wth the sldng wndow approach Computatonally expensve there are too many wndows multply by scales multply by aspect rato!!!! Need very fast classfers Typcally lmted to lnear SVMs and boostng But these are not the most accurate (kernel SVMs, etc) 13

14 Intellgent sldng wndows Instead of exhaustvely searchng over all possble wndows, lets ntellgently choose locatons where the classfer s evaluated Some consderatons: We want a small number of such regons (~1000) We want hgh recall no objects should be mssed Category ndependent - that way we can share the cost of computng features Fast shouldn t be slower than runnng the detector tself 14

15 How do we get such proposals? Why mght ths be a good dea? Can use low-level cues such as color and texture smlarty whch are category ndependent Often fast to compute Segmentatons Inherently span scale and aspect-rato Recognton usng regons, Gu et al. 15

16 We wll look at ths approach Segmentaton as Selectve Search for Object Recognton, K. Van de Sande, J. Ujlngs, T. Gevers, and A. Smeulders, ICCV 2013 Wnner of the PASCAL VOC challenge n recent years 16

17 Lets start wth segmentatons We typcally get over-segmentaton for bg objects,.e., objects are broken nto multple regons How can we fx ths? Effcent graph-based mage segmentaton Felzenszwalb and Huttenlocher, IJCV

18 Images are ntrnscally herarchcal!!!!!! How to obtan hgh recall? Segmentaton at a sngle scale s not enough Lets merge regons to produce a herarchy 18

19 Compute smlarty measure between all adjacent regon pars a and b as: Herarchcal clusterng 19

20 Herarchcal clusterng 1.Merge two most smlar regons based on S 2.Update smlartes between the new regon and ts neghbors 3.Go back to step 1 untl the whole mage s a sngle regons 20

21 Example proposals 21

22 Example proposals 22

23 Addng dversty to the proposals No sngle segmentaton works for all mages Use dfferent color spaces RGB, Opponent color (e.g., LAB), normalzed rgb, hue Vary parameters n the Felzenszwalb segmentaton method k = [100, 150, 200, 250] (k= threshold parameter) 23

24 Evaluatng object proposals (, )= We want: 1. Every ground truth box be covered by at least one proposal 2. We want as few proposals as possble 24

25 Evaluatng object proposals Recall s the proporton of objects that are covered by some box wth overlap > 0.5 Compare ths to ~100,000 regons for sldng wndows 25

26 Another approach: Objectness" What s an object? Alexe et al., CVPR 2010 Learns to detect objects from background usng color, texture, edge cues generc object detector One of the early methods for object proposals 26

27 Another approach: Edge boxes Edge Boxes: Locatng Object Proposals from Edges, Ztnck and Dollar, ECCV 2014 Number of contours that are wholly contaned nsde the box s an ndcatve of the lkelhood that the box contans an object. Very fast (0.25s per mage) 27

28 Detecton usng regon proposals Once agan, detecton = repeated classfcaton But we only classfy object proposals Tranng a classfer 28

29 Detals of the features HOG was used n the Dalal & Trggs model for effcency But we can use complex features and better classfers In partcular SIFT bag of words features Image by Andrea Vedald 29

30 SVM classfer wth a hstogram ntersecton kernel Recap of SVMs Detals of the classfer 30

31 Lnear classfers 31

32 Fnd lnear functon (hyperplane) to separate postve and negatve examples Lnear classfers x x postve : negatve : x x w w + + b b < 0 0 Whch hyperplane s best? 32

33 Support vector machnes Fnd hyperplane that maxmzes the margn between the postve and negatve examples C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

34 Support vector machnes Fnd hyperplane that maxmzes the margn between the postve and negatve examples x x postve ( y negatve( y = 1) : = 1) : x x w w + b + b 1 1 For support vectors, x w + b = ± 1 Dstance between pont and hyperplane: w x w + b Support vectors Margn Therefore, the margn s 2 / w C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

35 Fndng the maxmum margn hyperplane 1. Maxmze margn 2 / w 2. Correctly classfy all tranng data: x x postve ( negatve( Quadratc optmzaton problem: y y = 1) : = 1) : x x w w + + b b 1 1 mn w, b 1 2 w 2 subject to y ( w x + b) 1 C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

36 Fndng the maxmum margn hyperplane Soluton: w = α y x Learned weght (nonzero only for support vectors) C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

37 Fndng the maxmum margn hyperplane Soluton: w = α y x w x +b = y, for any support vector Classfcaton functon (decson boundary): w x + b = α y x x + b Notce that t reles on an nner product between the test pont x and the support vectors x Solvng the optmzaton problem also nvolves computng the nner products x x j between all pars of tranng ponts C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

38 ! Separable:!! Non-separable:!!! C: tradeoff constant, ξ : slack varable (postve) Whenever margn s 1, ξ = 0 Whenever margn s < 1, What f the data s not lnearly separable? 38 1 ) ( subject to 2 1 mn 2, + b y b x w w w 0 1 ) ( subject to 2 1 mn 1 2, = n b b y C ξ ξ x w w w ) ( 1 b y + = x w ξ

39 What f the data s not lnearly separable? mn w, b 1 2 w 2 + C n = 1 max ( 0,1 y ( w x + b) ) Maxmze margn Mnmze classfcaton mstakes 39

40 What f the data s not lnearly separable? mn w, b 1 2 w 2 + C n = 1 max ( 0,1 y ( w x + b) ) Margn -1 0 Demo:

41 Nonlnear SVMs Datasets that are lnearly separable work out great:! 0 x But what f the dataset s just too hard?! 0 x We can map t to a hgher-dmensonal space: x 2 0 x Slde credt: Andrew Moore 41

42 Nonlnear SVMs General dea: the orgnal nput space can always be mapped to some hgher-dmensonal feature space where the tranng set s separable: Φ: x φ(x) Slde credt: Andrew Moore 42

43 The kernel trck: nstead of explctly computng the lftng transformaton φ(x), defne a kernel functon K such that! Nonlnear SVMs K(x, y) = φ(x) φ(y) (the kernel functon must satsfy Mercer s condton) Ths gves a nonlnear decson boundary n the orgnal feature space: α y ϕ( x ) ϕ( x) + b = α y K( x, x) + b C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,

44 Accuracy vs. Evaluaton Tme Evaluaton tme Lnear Kernel Non- lnear Kernel Accuracy Lnear SVM: O (feature dmenson) Non Lnear SVM: O ( # support vectors X feature dmenson) 44

45 What s the Intersecton Kernel? Hstogram Intersecton kernel between hstograms a, b: a b mn(a,b) Introduced by Swan and Ballard 1991 to compare color hstograms. 45

46 SVM classfcaton functon sum over support vectors #sv tmes slower than lnear SVM 46

47 Maj, Berg and Malk, CVPR 08 SVM classfcaton functon Key Insght : Addtve Property

48 Maj, Berg and Malk, CVPR 08 SVM classfcaton functon Algorthm 1

49 Maj, Berg and Malk, CVPR 08 SVM classfcaton functon Algorthm 1

50 SVM classfcaton functon Algorthm 1 sort the support vector values n each coordnate, and pre- compute these sums for each rank. Maj, Berg and Malk, CVPR 08

51 SVM classfcaton functon SVM classfcaton functon Algorthm 1 sort the support vector values n each coordnate, and pre- compute these sums for each rank. To evaluate, fnd poston of n the sorted support vector values (cost : log #sv) look up values, multply & add

52 For IK h s pecewse lnear, and qute smooth, blue plot. We can approxmate wth fewer unformly spaced segments, red plot. Saves tme & space! SVM classfcaton functon Algorthm 2

53 SVM classfcaton functon Algorthm 2 Intersecton Ch- squared ensen- Shannon

54 Maj, Berg and Malk, CVPR 08 Lnear vs. Intersecton Kernel SVM Dataset Measure Lnear SVM IK SVM Speedup INRIA pedestrans 2 FPPI X DC pedestrans Accuracy X Caltech101, 15 examples Accuracy X Caltech101, 30 examples Accuracy X MNIST dgts Error X UIUC cars (Sngle Scale) Precson@ EER X On average 5x slower than lnear SVM but x faster than standard kernel SVM classfer

55 Results on PASCAL VOC detecton PASCAL VOC 2010 detecton results Ths paper = selectve search Does better on deformable objects such as anmals 55

56 Current state of the art n detecton R-CNNs (Grshck et al.) Regons wth CNN features!!!!!!! We wll look at CNNs n the next lecture 56