Find that! Visual Object Detection Primer

Size: px

Start display at page:

Download "Find that! Visual Object Detection Primer"

Abel Rice
5 years ago
Views:

1 Find that! Visual Object Detection Primer SkTech/MIT Innovation Workshop August 16, 2012 Dr. Tomasz Malisiewicz

2 Find that! Your Goals...imagine one such system that drives information queries directly from images and objects in its surroundings... The task is to evaluate current performances and study what type of applications could be enabled by actual systems.

3 My Goal Technology advisor: guide you in the right direction, but you define the direction Analogy: You have formed a startup and hired me as an outside technology consultant I ll be available during Guided Project Time (schedule will be sent out soon)

4 Overview Part I: How state-of-the-art visual object detection systems works Part II: Guide you through the process of using a state-of-the art object detection system in MATLAB

5 Recognition Tasks Image Classification Does the image contain an aeroplane? Object Class Detection/Localization Where are the aeroplanes (if any)? Object Class Segmentation Which pixels are part of an aeroplane (if any)?

6 Recognition Task Object Class Detection/Localization Where are the aeroplanes (if any)? Challenges Imaging factors e.g. lighting, pose, occlusion, clutter Intra-class variation Compared to Classification Detailed prediction e.g. bounding box Location usually provided for training

7 Preview of typical results aeroplane bicycle car cow horse motorbike

8 Outline 1. Sliding window detectors 2. Features and adding spatial information 3. Histogram of Oriented Gradients (HOG) 4. Two state of the art algorithms and PASCAL VOC 5. The future and challenges

9 Detection by Classification Basic component: binary classifier Car/non-car Classifier Yes, No, not a a car car

10 Detection by Classification Detect objects in clutter by search Car/non-car Classifier Sliding window: exhaustive search over position and scale

11 Detection by Classification Detect objects in clutter by search Car/non-car Classifier Sliding window: exhaustive search over position and scale

12 Detection by Classification Detect objects in clutter by search Car/non-car Classifier Sliding window: exhaustive search over position and scale (can use same size window over a spatial pyramid of images)

13 Window (Image) Classification Training Data Feature Extraction Classifier Features usually engineered Classifier learnt from data Car/Non-car

14 Outline 1. Sliding window detectors 2. Features and adding spatial information 3. Histogram of Oriented Gradients + linear SVM classifier Dalal & Triggs pedestrian detector HOG and history Training an object detector 4. Two state of the art algorithms and PASCAL VOC 5. The future and challenges

15 Dalal & Triggs CVPR 2005 Pedestrian detection Objective: detect (localize) standing humans in an image Sliding window classifier Train a binary classifier on whether a window contains a standing person or not Histogram of Oriented Gradients (HOG) feature Although HOG + SVM originally introduced for pedestrians has been used very successfully for many object categories

pixel cells each cell represented by histogram over 8

16 Feature: Histogram of Oriented image Gradients (HOG) dominant direction HOG tile 64 x 128 pixel window into 8 x 8 pixel cells each cell represented by histogram over 8 orientation bins (i.e. angles in range degrees) frequency orientation

17 Window (Image) Classification Training Data Feature Extraction Classifier HOG Features Linear SVM classifier pedestrian/non-pedestrian

19 Averaged examples

20 Classifier: linear SVM Advantages of linear SVM:!!"" #!! " $ # Training (Learning) Very efficient packages for the linear case, e.g. LIBLINEAR for batch training and Pegasos for on-line training. Complexity O(N) for N training points (cf O(N^3) for general SVM) Testing (Detection) Non-linear Linear!!!" #!!!" # "! # # $!! # %!"$& "! # #! #!! $ & # "!! $ & S = # of support vectors = (worst case ) N size of training data Independent of size of training data More on linear/non-linear in the image classification lab

21 Dalal and Triggs, CVPR 2005

22 Learned model!!!" # "!! $ " average over positive training data

23 Slide from Deva Ramanan

24 Training a sliding window detector Object detection is inherently asymmetric: much more non-object than object data Classifier needs to have very low false positive rate Non-object category is very complex need lots of data

25 Object Detection with Discriminatively Trained Part Based Models Pedro F. Felzenszwalb, David Mcallester, Deva Ramanan, Ross Girshick PAMI 2010

26 Approach Mixture of deformable part-based models One component per aspect e.g. front/side view Each component has global template + deformable parts Discriminative training from bounding boxes alone

27 Example Model One component of person model x 1 x 6 x 2 x 5 x 3 x 4 root filters coarse resolution part filters finer resolution deformation models

28 Object Hypothesis Position of root + each part Each part: HOG filter (at higher resolution) ( = (!"$%%%$&!')!" : location of root!#$%%%$&!' : location of parts Score is sum of filter scores minus deformation costs

29 Training Training data = images + bounding boxes Need to learn: model structure, filters, deformation costs Training

30 Person Model root filters coarse resolution part filters finer resolution deformation models Handles partial occlusion/truncation

31 Person model with 3 left-right components Mixture model using max over multiple components with leftright pairs

32 Car Model root filters coarse resolution part filters finer resolution deformation models

33 Car Detections high scoring true positives high scoring false positives

34 Person Detections high scoring true positives high scoring false positives (not enough overlap)

35 Progress Results on 2008 data improve for best methods for almost all categories Caveats: More training data + re-use of test data

36 Quiz I show you learned models, you guess the category

37 Can you guess the object category? Bottle Chair Motorbike

38 Can you guess the object category? Bottle Chair Motorbike

39 Can you guess the object category? Bottle Chair Motorbike

40 Can you guess the object category? Bottle Chair Motorbike

41 Summary To localize objects inside images, object detectors must search within the image (sliding window detector) State-of-the-art features are based on Histograms of Oriented Gradients (HOG) You can start tinkering with pre-trained object models today!

42 Part II: Using a state-ofthe-art detector Download MATLAB code Go to LDPM homepage download latest code version

43 Run detector demo >> cd voc-release4.01/ Compile C++ functions >> compile Test by running demo >> demo

44 inside demo.m

45 What is possible You can train your own object models, but you ll need data NOTE: training requires running the detector many times on a large set of images and can take days

47 Your assignment 1.) Get the MATLAB code up and running 2.) Choose an object model and download images of that category from Google image search 3.) Run the detector on those images 4.) Show the top scoring detection inside each image 5.) Determine how long the detector takes to run for your images (try help tic within MATLAB)

Category-level localization

Category-level localization Cordelia Schmid Recognition Classification Object present/absent in an image Often presence of a significant amount of background clutter Localization / Detection Localize object