Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels

Transcription

1 Training-Free, Generic Object Detection Using Locally Adaptive Regression Kernels IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIENCE, VOL.32, NO.9, SEPTEMBER 2010 Hae Jong Seo, Student Member, IEEE Peyman Milanfar, Fellow, IEEE 1

2 Overview Introduction Detail of The Framework Theoretical Justification Handling Variations in Scale, Rotation and Color Image Experimental Results Conclusion 2

3 Training v.s Training-Free Introduction Problem Specification Proposed Approach 3

4 Training v.s Training-Free With only one query. Require no training phase. Make a real-time object detection system while achieving high detection rates. 4

5 Problem Specification Address the localization problem of searching for an object of interest with only a single query image. Learning-based methods require a large amount of training example and plenty of time. Focus on sophisticated feature such as histograms, gradients, and shape descriptors [16]. 5

6 Proposed Approach Use local regression kernels as descriptors, which capture local structure of the data well, even with distortions [17]. The fundamental component is the calculation of LSK. Use a nonparametric nearest neighbor classifier, along with a generalization of the cosine similarity. Propose Canonical Cosine Similarity to extend the framework to the case such as a color image. 6

7 Proposed Approach The key idea of LSK is to obtain data structure by analyzing pixel value differences based on gradient. Use the structure information above to determine the shape and size of a canonical kernel. Use Matrix Cosine Similarity as similarity measure. Adopt the idea of a significance and non-maximum suppression to deal with the case that no object in ROI. 7

8 Proposed Approach 8

9 Detail of The Framework Local Steering Kernel Descriptor Matrix Cosine Similarity Significance Test 9

10 Local Steering Kernel Descriptor Key idea of LSK is to obtain the local structure of images by analyzing pixel value differences based on estimated gradient. P 2 is the number of pixels in a local window (P*P). 10

11 Local Steering Kernel Descriptor 11

12 Local Steering Kernel Descriptor In this paper, steering matrices are chosen a Gaussian function for K(.). 12

13 Local Steering Kernel Descriptor Normalization of this kernel function yields invariance to brightness change and contrast change. 13

14 Local Steering Kernel Descriptor Feature Representation In order to organize normalized LSK mentioned at P.12, let W Q, W T be matrices whose columns are vectors w Q j, w T j. 14

15 Local Steering Kernel Descriptor Feature Representation W Q = w Q 1,, w Q n W Q = 1,1 W Q 1,p^2 W Q W Q n,1 W Q n,p^2 w Q 1 w Q a w Q b In our current program and test case, n = 128*64 p = 3. 15

16 Local Steering Kernel Descriptor Feature Representation By applying PCA to W Q, we can retain the largest d principal components A Q. The lower dimensional features are computed by projecting W Q and W T onto A Q : 16

17 Local Steering Kernel Descriptor Feature Representation Fig. 6. Face and car examples: (a) A Q learned from a collection of LSKs W Q, (b) feature row vectors of F Q from query Q, (c) feature row vectors F T from target image T. Eigenvectors and feature vectors were reshaped into image and up-scaled for illustration purpose. 17

18 Matrix Cosine Similarity This step is a decision rule based on the measurement of a distance between the computed features F Q, F T. The cosine similarity is defined as the inner product between two normalized vectors. 18

19 Matrix Cosine Similarity If we deal with the features F Q, F Ti, that consist of a set of vectors, Matrix Cosine Similarity can be defined as a natural generalization using the Frobenius inner product between two normalized matrices. 19

20 Matrix Cosine Similarity And the equation mentioned above can be rewritten as follows: RM : f ρ i = ρ i 2 1 ρ i 2 20

21 Matrix Cosine Similarity Resemblance Map 21

22 Significance Test To avoid the case that there may not be any object of interest in the target image. With non-maximum suppression, the number of object can be found. And it can avoid detection the same object again as well. 22

23 Theoretical Justification 23

24 Pseudo code for the nonparametric object detection algorithm Stage 1: Feature representation 1) Construct W Q, W T. (a collection of normalized LSK) 2) Apply PCA to W Q to get A Q. 3) Project W Q, W T onto A Q to get F Q and F T. 24

25 Pseudo code for the nonparametric object detection algorithm Stage 2: Compute Matrix Cosine Similarity for every target patch T i, where i = 0 ~ M-1 do MCS (mentioned at P.20) end for Then, find max f ρ i. 25

26 Pseudo code for the nonparametric object detection algorithm Stage 3: Significance tests and Non-maximum suppression 1) If max f ρ i > τ 0, go to the next test. Otherwise, there is no object of interest in T. 2) Threshold RM by τ which is set to achieve 99 percent confidence level from the empirical PDF of f ρ i. 3) Apply non-maximum suppression to RM until the local maximum value is below τ. 26

27 Handling Variations Multiscale Multirotation Canonical Cosine Similarity 27

28 Handling Variations The framework mentioned above only deals with the detection of objects in a gray image at a single scale. Rotation and scale variations should be handled as well. Color images should be also considered. 28

29 Multiscale Approach 29

30 Multirotation Approach 30

31 Canonical Cosine Similarity Suppose that at each pixel the image has q values. Generate q feature sets F l l Q, F Ti, l = 1 ~ q. The key idea is to find vectors U Q and U T, which maximally correlate two data sets F Q, F Ti 31

32 Canonical Cosine Similarity Then, the objective function we are maximizing is the cosine similarity as follows: Where 32

33 Canonical Cosine Similarity U Q and U Ti mentioned above are called canonical variates and can be obtained by solving the coupled eigenvalue problems as follows : 33

34 Experimental Results Car Detection Face Detection General Object Detection 34

35 Experiment Result If the detected region by the proposed method lies within an ellipse of a certain size center around the ground truth, it will be evaluated as a correct detection. TP is the number of true positives, FP is the number of false positives, np is the total number of positives in the data set. 35

36 Car Detection The UIUC car data set consists of learning and test sets. The set contains 550 positive images and 500 negative images. 36

37 Car Detection (a) Examples of correct detections on the UIUC single-scale car test set [53]. 37

38 Car Detection (b) Examples of correct detections on the UIUC multi-scale car test set. 38

39 Comparison With the method without PCA 39

40 Comparison Between the Proposed Method and Other Methods 40

41 Face Detection A test set is chosen from a subset of the MIT-CMU face data set [54]. The test set is composed of 43 gray-scale images containing 149 frontal faces at various sizes and 20 gray-scale images containing 30 faces with various rotations. 41

42 Face Detection 42

43 Face Detection 43

44 Face Detection Fig. 20. Left: precision-recall curves. Right: ROC curves on the MIT-CMU test set [54] using two different query images. 44

45 General Object Detection 45

46 Conclustion 46

47 Conclusion Unlike other learning-based detection methods, the proposed framework operate using a single example to find similar match, doesn t require any prior knowledge and any preprocessing step of the target image. Extension of the method to a large-scale data set requires a significant improvement. Doesn t use the color information. 47