Beyond Bags of Features

Size: px
Start display at page:

Download "Beyond Bags of Features"

Transcription

1 : for Recognizing Natural Scene Categories Matching and Modeling Seminar Instructed by Prof. Haim J. Wolfson School of Computer Science Tel Aviv University December 9 th, 2015

2 Outline Pyramid Matching Kernels Spatial Matching Scheme 4

3 Outline Pyramid Matching Kernels Spatial Matching Scheme 4

4 We want to recognize semantic category of an image Goals: Scene (forest, street, etc.) Objects of interest Efficiency Accuracy Based on [Lazebnik et al., 2006]

5 Outline Pyramid Matching Kernels Spatial Matching Scheme 4

6 Set of methods for identification and classification Usually: Define image features For example, SIFT descriptors in image blocks Orderless representation of (local) image features Usually a long vector For example, histogram with k-bins Learning algorithm for classification or detection Such as SVM

7 Set of methods for identification and classification Usually: Define image features For example, SIFT descriptors in image blocks Orderless representation of (local) image features Usually a long vector For example, histogram with k-bins Learning algorithm for classification or detection Such as SVM

8 Set of methods for identification and classification Usually: Define image features For example, SIFT descriptors in image blocks Orderless representation of (local) image features Usually a long vector For example, histogram with k-bins Learning algorithm for classification or detection Such as SVM

9 Disadvantages Existing methods usually use local features Hence, they disregard spatial layout of the feature For example: Incapability of capturing shape of objects Incapability of segmenting an object from its background Overcoming these limitations is challenging Especially with noise, such as: Clutter Occlusion Viewpoint change

10 Overcoming the Disadvantages Previous works suggest: Robustness at significant computational expense Efficiency with inconclusive results In this paper: Statistics of local features over fixed subregions

11 Off Topic: SVM Support Vector Machine Supervised learning algorithm Creates a (binary) linear classifier Finds hyperplane to separate the categories Has lot of extensions For example, multiclass SVM

12 Off Topic: SVM More Formally Given D = { (x i, y i ) x i R d, y i {±1} } Find w, b that maximizes (x i,y i ) D y i (w x i b) Given a new example x, its classification y satisfies: { 1 w x b > 0 y = 1 w x b 0 w x b = 0 is called the separating hyperplane

13 Outline Pyramid Matching Kernels Spatial Matching Scheme Pyramid Matching Kernels Spatial Matching Scheme 4

14 Pyramid Matching Kernels Spatial Matching Scheme Kernel based recognition method Pyramid matching scheme of Grauman and Darrell Repeatedly subdivide the image For each division, compute histograms of local features

15 Pyramid Matching Kernels Pyramid Matching Kernels Spatial Matching Scheme Let X and Y be two sets of d-dimensional vectors Vectors in a feature space We want to find an approximate correspondence in them

16 Resolutions Pyramid Matching Kernels Spatial Matching Scheme Construct a sequence of grids for resolutions 0,..., L The grid of level l has 2 l cells along each dimension Total of D l = 2 dl cells Let HX l and Hl Y resolution denote the histograms of X and Y in this Let HX l (i) and Hl Y (i) be the numbers of points from X and Y in the i th cell

17 Matchings in Resolution Pyramid Matching Kernels Spatial Matching Scheme If x X satisfies x H l X (i), then x matches Hl X (i) The number of matches at level l is: I l = I I l also includes I l+1 ( ) HX, l HY l = D l i=1 { } min HX l (i), HY l (i) Therefore, the number of new matches of level l is I l I l+1 for 0 l < L

18 Pyramid Match Kernel Pyramid Matching Kernels Spatial Matching Scheme Give level l the weight 1 2 L l The pyramid match kerenl is defined as: L 1 κ L (X, Y ) = I L 1 + (I 2 L l l I l+1) l=0 = 1 2 L I0 + L l=1 2 1 L l+1 Il

19 Spatial Information Pyramid Matching Kernels Spatial Matching Scheme So far, we have counted the number of features in each cell But we ignored that the features might differ or be similar Now we will quantize all feature vectors into M discrete types Only features of the same type can match one another Each channel m gives us two sets of vectors, X m and Y m This approach reduces to standard bag of features when L = 0

20 Spatial Information Pyramid Matching Kernels Spatial Matching Scheme So far, we have counted the number of features in each cell But we ignored that the features might differ or be similar Now we will quantize all feature vectors into M discrete types Only features of the same type can match one another Each channel m gives us two sets of vectors, X m and Y m This approach reduces to standard bag of features when L = 0

21 The Image Kernel Pyramid Matching Kernels Spatial Matching Scheme The image kernel m K L (X, Y ) = κ L (X m, Y m ) m=1

22 Vector Length Pyramid Matching Kernels Spatial Matching Scheme Our images are two dimensional (d = 2) Hence each resolution divides the previous one by 2 2 = 4 For L levels and M channels, the vector length is: M L l=0 4 l = M 4L+1 1 3

23 Efficient Implementation Pyramid Matching Kernels Spatial Matching Scheme 1 Subdivide the image to L levels 2 For each level and each channe: 1 Count the features that fall in each spatial bin 2 Weight this spatial bin, and add its weighted value to the histogram The next slide shows an example

24 Efficient Implementation Pyramid Matching Kernels Spatial Matching Scheme 1 Subdivide the image to L levels 2 For each level and each channe: 1 Count the features that fall in each spatial bin 2 Weight this spatial bin, and add its weighted value to the histogram The next slide shows an example

25 Example Pyramid Matching Kernels Spatial Matching Scheme + + level level level /4 1/4 1/2

26 Outline Pyramid Matching Kernels Spatial Matching Scheme 4

27 About the Experiments Each experiment was repeated 10 times Each with different randomly selected training and testing sets The average of per-class is recorded for each run The final result consists of mean and standard deviation Multiclass classification was done using SVM Trained using ove vs. all rule The label is assigned by the classifier with the highest response

28 Used Features Two kinds of features: Weak features Strong features

29 The Weak Features Oriented edge points Extract edfe points at two scales and eight orientations Total of M = 16 channels

30 The Strong Features SIFT descriptors pixel batches, grid of 8 8 pixels Visual vocabulary was calculated by: k-means clustering of random subset of patches Taken from the training set Used with M = 200 or M = 400

31 Dataset consists of 15 scene categories Each category has 200 to 400 images Average image size is pixels Originally from [Li and Perona, 2005] with 13 categories 2 categories were collected by [Lazebnik et al., 2006]

32 Dataset Sample

33 Results for Classification Results Weak features (M = 16) Strong Features (M = 200) Strong Features (M = 400) L Single-level Pyramid Single-level Pyramid Single-level Pyramid 0 (1 1) 45.3 ± ± ± (2 2) 53.6 ± ± ± ± ± ± (4 4) 61.7 ± ± ± ± ± ± (8 8) 63.3 ± ± ± ± ± ± 0.6

34 Results Analysis Results improve dramatically as we go from L = 0 to multilevel For strong features, performance drop from L = 2 to L = 3 Because the highest level of L = 3 is too finely subdivided Individual bins yielding too few matches Still small performance drop Strong features are better than weak features However, the number of strong features is almost irrelevant

35 Confusion Table

36 Confusions Confusions occurs between: Indoor classes (kitchen, bedroom, living room) Natural classes (coast, open country) This is understandable Next slide shows example of image retrieval

37 Image Retrieval Examples

38 Previous Work [Li and Perona, 2005] achieved a success rate of 65.2% with L = 0 and M = 200 Whereas [Lazebnik et al., 2006] achieved 72.2% 74.7% for the 13 categories of [Li and Perona, 2005] [Lazebnik et al., 2006] wondered why the gap is so big Apparently, [Li and Perona, 2005] used worse strong features than [Lazebnik et al., 2006] [Lazebnik et al., 2006] achieved 65.9% with the features of [Li and Perona, 2005]

39 Based on [Fei-Fei et al., 2004] 101 categories 31 to 800 images per category Image size about pixels Properties: Relatively little clutter Objects are centered Objects occupy most of the image

40 Experiment Setup Train 30 images per class Test on the rest For efficiency, limit test to 50 images per class Same setup as in [Grauman and Darrell, 2005, Zhang et al., 2007].

41 Results Weak features Strong features (M = 200) L Single-level Pyramid Single-level Pyramid ± ± ± ± ± ± ± ± ± ± ± ± ± ± 0.7 Results for M = 400 do not improve, just as for scenes

42 Results Examples Easiest vs. Hardest Categories Highest performance, on L = 2, M = 200

43 Comparison to Previous Work For L = 0, [Grauman and Darrell, 2005] got 43% We ve got 41.2% Our best result is 64.6%, when L = 2, M = 200 Better than 53.9% by [Zhang et al., 2007]

44 Confusions There are lot of closely related classes Top five confusions for L = 2, M = 200: Class 1 Class 2 Class 1 Class 2 misclassified misclassified as class 2 as class 1 ketch schooner lotus water lily crocodile crocodile head crayfish lobster flamingo ibis

45 So Far Our method does well assuming canonical poses What about: Heavy clutter? Pose changes?

46 [Opelt et al., 2004] Characterized by high intra-class cariation Two object classes: Objects: Bikes (373 images) + Persons (460 images) Backgrounds (270 images) Image resolution is pixels Range of scale and poses is very diverse For example, in the persons category: Pedestrian in the distance Side view of a complete body Closeup of a head

47 Examples Bikes

48 Examples Persons

49 Examples Backgrounds

50 Experiment Setup Same as in [Opelt et al., 2004]: Two class detection: objects vs. background Train on 100 positive and 100 negative images 50 from the other object class 50 from the backgrounds class Test on similarly distributing set

51 Results Strong Features, M = 200 Class L = 0 L = 2 Opelt Zhang Bikes 82.4 ± ± People 79.5 ± ± Standard deviation is quite high For example, the bikes class for L = 2 ranges from 81% to 91% Still we are close to other methods

52 Bibliography Bibliography I Fei-Fei, L., Fergus, R., and Perona, P. (2004). Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In Computer Vision and Pattern Recognition Workshop, CVPRW 04. Conference on, pages Grauman, K. and Darrell, T. (2005). The pyramid match kernel: discriminative classification with sets of image features. In Computer Vision, ICCV Tenth IEEE International Conference on, volume 2, pages Vol. 2.

53 Bibliography Bibliography II Lazebnik, S., Schmid, C., and Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2, CVPR 06, pages , Washington, DC, USA. IEEE Computer Society. Li, F.-F. and Perona, P. (2005). A bayesian hierarchical model for learning natural scene categories. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 05) -

54 Bibliography Bibliography III Volume 2 - Volume 02, CVPR 05, pages , Washington, DC, USA. IEEE Computer Society. Opelt, A., Fussenegger, M., Pinz, A., and Auer, P. (2004). Weak hypotheses and boosting for generic object detection and recognition. In Pajdla, T. and Matas, J., editors, Computer Vision - ECCV 2004, volume 3022 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg. Zhang, J., Marszalek, M., Lazebnik, S., and Schmid, C. (2007). Local features and kernels for classification of texture and object categories: A comprehensive study. Int. J. Comput. Vision, 73(2):

Beyond bags of Features

Beyond bags of Features Beyond bags of Features Spatial Pyramid Matching for Recognizing Natural Scene Categories Camille Schreck, Romain Vavassori Ensimag December 14, 2012 Schreck, Vavassori (Ensimag) Beyond bags of Features

More information

Previously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011

Previously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011 Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition

More information

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce CVR-TR-2005-04 Abstract This paper presents a method for recognizing

More information

Part-based and local feature models for generic object recognition

Part-based and local feature models for generic object recognition Part-based and local feature models for generic object recognition May 28 th, 2015 Yong Jae Lee UC Davis Announcements PS2 grades up on SmartSite PS2 stats: Mean: 80.15 Standard Dev: 22.77 Vote on piazza

More information

Artistic ideation based on computer vision methods

Artistic ideation based on computer vision methods Journal of Theoretical and Applied Computer Science Vol. 6, No. 2, 2012, pp. 72 78 ISSN 2299-2634 http://www.jtacs.org Artistic ideation based on computer vision methods Ferran Reverter, Pilar Rosado,

More information

Beyond bags of features: Adding spatial information. Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba

Beyond bags of features: Adding spatial information. Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba Adding spatial information Forming vocabularies from pairs of nearby features doublets

More information

Object Classification Problem

Object Classification Problem HIERARCHICAL OBJECT CATEGORIZATION" Gregory Griffin and Pietro Perona. Learning and Using Taxonomies For Fast Visual Categorization. CVPR 2008 Marcin Marszalek and Cordelia Schmid. Constructing Category

More information

Patch Descriptors. CSE 455 Linda Shapiro

Patch Descriptors. CSE 455 Linda Shapiro Patch Descriptors CSE 455 Linda Shapiro How can we find corresponding points? How can we find correspondences? How do we describe an image patch? How do we describe an image patch? Patches with similar

More information

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped

More information

Bag-of-features. Cordelia Schmid

Bag-of-features. Cordelia Schmid Bag-of-features for category classification Cordelia Schmid Visual search Particular objects and scenes, large databases Category recognition Image classification: assigning a class label to the image

More information

Part based models for recognition. Kristen Grauman

Part based models for recognition. Kristen Grauman Part based models for recognition Kristen Grauman UT Austin Limitations of window-based models Not all objects are box-shaped Assuming specific 2d view of object Local components themselves do not necessarily

More information

Beyond bags of features: spatial pyramid matching for recognizing natural scene categories

Beyond bags of features: spatial pyramid matching for recognizing natural scene categories Beyond bags of features: spatial pyramid matching for recognizing natural scene categories Svetlana Lazebnik, Cordelia Schmid, Jean Ponce To cite this version: Svetlana Lazebnik, Cordelia Schmid, Jean

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 16: Bag-of-words models Object Bag of words Announcements Project 3: Eigenfaces due Wednesday, November 11 at 11:59pm solo project Final project presentations:

More information

Object Recognition. Computer Vision. Slides from Lana Lazebnik, Fei-Fei Li, Rob Fergus, Antonio Torralba, and Jean Ponce

Object Recognition. Computer Vision. Slides from Lana Lazebnik, Fei-Fei Li, Rob Fergus, Antonio Torralba, and Jean Ponce Object Recognition Computer Vision Slides from Lana Lazebnik, Fei-Fei Li, Rob Fergus, Antonio Torralba, and Jean Ponce How many visual object categories are there? Biederman 1987 ANIMALS PLANTS OBJECTS

More information

CPPP/UFMS at ImageCLEF 2014: Robot Vision Task

CPPP/UFMS at ImageCLEF 2014: Robot Vision Task CPPP/UFMS at ImageCLEF 2014: Robot Vision Task Rodrigo de Carvalho Gomes, Lucas Correia Ribas, Amaury Antônio de Castro Junior, Wesley Nunes Gonçalves Federal University of Mato Grosso do Sul - Ponta Porã

More information

CLASSIFICATION Experiments

CLASSIFICATION Experiments CLASSIFICATION Experiments January 27,2015 CS3710: Visual Recognition Bhavin Modi Bag of features Object Bag of words 1. Extract features 2. Learn visual vocabulary Bag of features: outline 3. Quantize

More information

Improved Spatial Pyramid Matching for Image Classification

Improved Spatial Pyramid Matching for Image Classification Improved Spatial Pyramid Matching for Image Classification Mohammad Shahiduzzaman, Dengsheng Zhang, and Guojun Lu Gippsland School of IT, Monash University, Australia {Shahid.Zaman,Dengsheng.Zhang,Guojun.Lu}@monash.edu

More information

Introduction to object recognition. Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others

Introduction to object recognition. Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others Introduction to object recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and others Overview Basic recognition tasks A statistical learning approach Traditional or shallow recognition

More information

Preliminary Local Feature Selection by Support Vector Machine for Bag of Features

Preliminary Local Feature Selection by Support Vector Machine for Bag of Features Preliminary Local Feature Selection by Support Vector Machine for Bag of Features Tetsu Matsukawa Koji Suzuki Takio Kurita :University of Tsukuba :National Institute of Advanced Industrial Science and

More information

Beyond Bags of features Spatial information & Shape models

Beyond Bags of features Spatial information & Shape models Beyond Bags of features Spatial information & Shape models Jana Kosecka Many slides adapted from S. Lazebnik, FeiFei Li, Rob Fergus, and Antonio Torralba Detection, recognition (so far )! Bags of features

More information

Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition

Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition Mining Discriminative Adjectives and Prepositions for Natural Scene Recognition Bangpeng Yao 1, Juan Carlos Niebles 2,3, Li Fei-Fei 1 1 Department of Computer Science, Princeton University, NJ 08540, USA

More information

TA Section: Problem Set 4

TA Section: Problem Set 4 TA Section: Problem Set 4 Outline Discriminative vs. Generative Classifiers Image representation and recognition models Bag of Words Model Part-based Model Constellation Model Pictorial Structures Model

More information

Improving Recognition through Object Sub-categorization

Improving Recognition through Object Sub-categorization Improving Recognition through Object Sub-categorization Al Mansur and Yoshinori Kuno Graduate School of Science and Engineering, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama-shi, Saitama 338-8570,

More information

arxiv: v3 [cs.cv] 3 Oct 2012

arxiv: v3 [cs.cv] 3 Oct 2012 Combined Descriptors in Spatial Pyramid Domain for Image Classification Junlin Hu and Ping Guo arxiv:1210.0386v3 [cs.cv] 3 Oct 2012 Image Processing and Pattern Recognition Laboratory Beijing Normal University,

More information

Patch Descriptors. EE/CSE 576 Linda Shapiro

Patch Descriptors. EE/CSE 576 Linda Shapiro Patch Descriptors EE/CSE 576 Linda Shapiro 1 How can we find corresponding points? How can we find correspondences? How do we describe an image patch? How do we describe an image patch? Patches with similar

More information

String distance for automatic image classification

String distance for automatic image classification String distance for automatic image classification Nguyen Hong Thinh*, Le Vu Ha*, Barat Cecile** and Ducottet Christophe** *University of Engineering and Technology, Vietnam National University of HaNoi,

More information

Announcements. Recognition. Recognition. Recognition. Recognition. Homework 3 is due May 18, 11:59 PM Reading: Computer Vision I CSE 152 Lecture 14

Announcements. Recognition. Recognition. Recognition. Recognition. Homework 3 is due May 18, 11:59 PM Reading: Computer Vision I CSE 152 Lecture 14 Announcements Computer Vision I CSE 152 Lecture 14 Homework 3 is due May 18, 11:59 PM Reading: Chapter 15: Learning to Classify Chapter 16: Classifying Images Chapter 17: Detecting Objects in Images Given

More information

By Suren Manvelyan,

By Suren Manvelyan, By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan,

More information

Supervised learning. y = f(x) function

Supervised learning. y = f(x) function Supervised learning y = f(x) output prediction function Image feature Training: given a training set of labeled examples {(x 1,y 1 ),, (x N,y N )}, estimate the prediction function f by minimizing the

More information

Bag of Words Models. CS4670 / 5670: Computer Vision Noah Snavely. Bag-of-words models 11/26/2013

Bag of Words Models. CS4670 / 5670: Computer Vision Noah Snavely. Bag-of-words models 11/26/2013 CS4670 / 5670: Computer Vision Noah Snavely Bag-of-words models Object Bag of words Bag of Words Models Adapted from slides by Rob Fergus and Svetlana Lazebnik 1 Object Bag of words Origin 1: Texture Recognition

More information

Deformable Part Models

Deformable Part Models CS 1674: Intro to Computer Vision Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 9, 2016 Today: Object category detection Window-based approaches: Last time: Viola-Jones

More information

Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study

Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study J. Zhang 1 M. Marszałek 1 S. Lazebnik 2 C. Schmid 1 1 INRIA Rhône-Alpes, LEAR - GRAVIR Montbonnot, France

More information

Efficient Kernels for Identifying Unbounded-Order Spatial Features

Efficient Kernels for Identifying Unbounded-Order Spatial Features Efficient Kernels for Identifying Unbounded-Order Spatial Features Yimeng Zhang Carnegie Mellon University yimengz@andrew.cmu.edu Tsuhan Chen Cornell University tsuhan@ece.cornell.edu Abstract Higher order

More information

Visual Object Recognition

Visual Object Recognition Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial Visual Object Recognition Bastian Leibe Computer Vision Laboratory ETH Zurich Chicago, 14.07.2008 & Kristen Grauman Department

More information

ROBUST SCENE CLASSIFICATION BY GIST WITH ANGULAR RADIAL PARTITIONING. Wei Liu, Serkan Kiranyaz and Moncef Gabbouj

ROBUST SCENE CLASSIFICATION BY GIST WITH ANGULAR RADIAL PARTITIONING. Wei Liu, Serkan Kiranyaz and Moncef Gabbouj Proceedings of the 5th International Symposium on Communications, Control and Signal Processing, ISCCSP 2012, Rome, Italy, 2-4 May 2012 ROBUST SCENE CLASSIFICATION BY GIST WITH ANGULAR RADIAL PARTITIONING

More information

CS 231A Computer Vision (Fall 2011) Problem Set 4

CS 231A Computer Vision (Fall 2011) Problem Set 4 CS 231A Computer Vision (Fall 2011) Problem Set 4 Due: Nov. 30 th, 2011 (9:30am) 1 Part-based models for Object Recognition (50 points) One approach to object recognition is to use a deformable part-based

More information

Selection of Scale-Invariant Parts for Object Class Recognition

Selection of Scale-Invariant Parts for Object Class Recognition Selection of Scale-Invariant Parts for Object Class Recognition Gy. Dorkó and C. Schmid INRIA Rhône-Alpes, GRAVIR-CNRS 655, av. de l Europe, 3833 Montbonnot, France fdorko,schmidg@inrialpes.fr Abstract

More information

Object Category Detection. Slides mostly from Derek Hoiem

Object Category Detection. Slides mostly from Derek Hoiem Object Category Detection Slides mostly from Derek Hoiem Today s class: Object Category Detection Overview of object category detection Statistical template matching with sliding window Part-based Models

More information

Recognition with Bag-ofWords. (Borrowing heavily from Tutorial Slides by Li Fei-fei)

Recognition with Bag-ofWords. (Borrowing heavily from Tutorial Slides by Li Fei-fei) Recognition with Bag-ofWords (Borrowing heavily from Tutorial Slides by Li Fei-fei) Recognition So far, we ve worked on recognizing edges Now, we ll work on recognizing objects We will use a bag-of-words

More information

Local Features and Bag of Words Models

Local Features and Bag of Words Models 10/14/11 Local Features and Bag of Words Models Computer Vision CS 143, Brown James Hays Slides from Svetlana Lazebnik, Derek Hoiem, Antonio Torralba, David Lowe, Fei Fei Li and others Computer Engineering

More information

Local Features based Object Categories and Object Instances Recognition

Local Features based Object Categories and Object Instances Recognition Local Features based Object Categories and Object Instances Recognition Eric Nowak Ph.D. thesis defense 17th of March, 2008 1 Thesis in Computer Vision Computer vision is the science and technology of

More information

Distances and Kernels. Motivation

Distances and Kernels. Motivation Distances and Kernels Amirshahed Mehrtash Motivation How similar? 1 Problem Definition Designing a fast system to measure the similarity il it of two images. Used to categorize images based on appearance.

More information

Combining Selective Search Segmentation and Random Forest for Image Classification

Combining Selective Search Segmentation and Random Forest for Image Classification Combining Selective Search Segmentation and Random Forest for Image Classification Gediminas Bertasius November 24, 2013 1 Problem Statement Random Forest algorithm have been successfully used in many

More information

Aggregating Descriptors with Local Gaussian Metrics

Aggregating Descriptors with Local Gaussian Metrics Aggregating Descriptors with Local Gaussian Metrics Hideki Nakayama Grad. School of Information Science and Technology The University of Tokyo Tokyo, JAPAN nakayama@ci.i.u-tokyo.ac.jp Abstract Recently,

More information

Effective Classifiers for Detecting Objects

Effective Classifiers for Detecting Objects Effective Classifiers for Detecting Objects Michael Mayo Dept. of Computer Science University of Waikato Private Bag 3105, Hamilton, New Zealand mmayo@cs.waikato.ac.nz Abstract Several state-of-the-art

More information

Recap Image Classification with Bags of Local Features

Recap Image Classification with Bags of Local Features Recap Image Classification with Bags of Local Features Bag of Feature models were the state of the art for image classification for a decade BoF may still be the state of the art for instance retrieval

More information

Object recognition. Methods for classification and image representation

Object recognition. Methods for classification and image representation Object recognition Methods for classification and image representation Credits Slides by Pete Barnum Slides by FeiFei Li Paul Viola, Michael Jones, Robust Realtime Object Detection, IJCV 04 Navneet Dalal

More information

SEMANTIC SEGMENTATION AS IMAGE REPRESENTATION FOR SCENE RECOGNITION. Ahmed Bassiouny, Motaz El-Saban. Microsoft Advanced Technology Labs, Cairo, Egypt

SEMANTIC SEGMENTATION AS IMAGE REPRESENTATION FOR SCENE RECOGNITION. Ahmed Bassiouny, Motaz El-Saban. Microsoft Advanced Technology Labs, Cairo, Egypt SEMANTIC SEGMENTATION AS IMAGE REPRESENTATION FOR SCENE RECOGNITION Ahmed Bassiouny, Motaz El-Saban Microsoft Advanced Technology Labs, Cairo, Egypt ABSTRACT We introduce a novel approach towards scene

More information

Discriminative classifiers for image recognition

Discriminative classifiers for image recognition Discriminative classifiers for image recognition May 26 th, 2015 Yong Jae Lee UC Davis Outline Last time: window-based generic object detection basic pipeline face detection with boosting as case study

More information

Recognize Complex Events from Static Images by Fusing Deep Channels Supplementary Materials

Recognize Complex Events from Static Images by Fusing Deep Channels Supplementary Materials Recognize Complex Events from Static Images by Fusing Deep Channels Supplementary Materials Yuanjun Xiong 1 Kai Zhu 1 Dahua Lin 1 Xiaoou Tang 1,2 1 Department of Information Engineering, The Chinese University

More information

A Survey on Image Classification using Data Mining Techniques Vyoma Patel 1 G. J. Sahani 2

A Survey on Image Classification using Data Mining Techniques Vyoma Patel 1 G. J. Sahani 2 IJSRD - International Journal for Scientific Research & Development Vol. 2, Issue 10, 2014 ISSN (online): 2321-0613 A Survey on Image Classification using Data Mining Techniques Vyoma Patel 1 G. J. Sahani

More information

Learning Realistic Human Actions from Movies

Learning Realistic Human Actions from Movies Learning Realistic Human Actions from Movies Ivan Laptev*, Marcin Marszałek**, Cordelia Schmid**, Benjamin Rozenfeld*** INRIA Rennes, France ** INRIA Grenoble, France *** Bar-Ilan University, Israel Presented

More information

Learning Representations for Visual Object Class Recognition

Learning Representations for Visual Object Class Recognition Learning Representations for Visual Object Class Recognition Marcin Marszałek Cordelia Schmid Hedi Harzallah Joost van de Weijer LEAR, INRIA Grenoble, Rhône-Alpes, France October 15th, 2007 Bag-of-Features

More information

Loose Shape Model for Discriminative Learning of Object Categories

Loose Shape Model for Discriminative Learning of Object Categories Loose Shape Model for Discriminative Learning of Object Categories Margarita Osadchy and Elran Morash Computer Science Department University of Haifa Mount Carmel, Haifa 31905, Israel rita@cs.haifa.ac.il

More information

Max-Margin Dictionary Learning for Multiclass Image Categorization

Max-Margin Dictionary Learning for Multiclass Image Categorization Max-Margin Dictionary Learning for Multiclass Image Categorization Xiao-Chen Lian 1, Zhiwei Li 3, Bao-Liang Lu 1,2, and Lei Zhang 3 1 Dept. of Computer Science and Engineering, Shanghai Jiao Tong University,

More information

Bias-Variance Trade-off + Other Models and Problems

Bias-Variance Trade-off + Other Models and Problems CS 1699: Intro to Computer Vision Bias-Variance Trade-off + Other Models and Problems Prof. Adriana Kovashka University of Pittsburgh November 3, 2015 Outline Support Vector Machines (review + other uses)

More information

Comparing Local Feature Descriptors in plsa-based Image Models

Comparing Local Feature Descriptors in plsa-based Image Models Comparing Local Feature Descriptors in plsa-based Image Models Eva Hörster 1,ThomasGreif 1, Rainer Lienhart 1, and Malcolm Slaney 2 1 Multimedia Computing Lab, University of Augsburg, Germany {hoerster,lienhart}@informatik.uni-augsburg.de

More information

Action recognition in videos

Action recognition in videos Action recognition in videos Cordelia Schmid INRIA Grenoble Joint work with V. Ferrari, A. Gaidon, Z. Harchaoui, A. Klaeser, A. Prest, H. Wang Action recognition - goal Short actions, i.e. drinking, sit

More information

Visual words. Map high-dimensional descriptors to tokens/words by quantizing the feature space.

Visual words. Map high-dimensional descriptors to tokens/words by quantizing the feature space. Visual words Map high-dimensional descriptors to tokens/words by quantizing the feature space. Quantize via clustering; cluster centers are the visual words Word #2 Descriptor feature space Assign word

More information

Recognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213)

Recognition of Animal Skin Texture Attributes in the Wild. Amey Dharwadker (aap2174) Kai Zhang (kz2213) Recognition of Animal Skin Texture Attributes in the Wild Amey Dharwadker (aap2174) Kai Zhang (kz2213) Motivation Patterns and textures are have an important role in object description and understanding

More information

A 2-D Histogram Representation of Images for Pooling

A 2-D Histogram Representation of Images for Pooling A 2-D Histogram Representation of Images for Pooling Xinnan YU and Yu-Jin ZHANG Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China ABSTRACT Designing a suitable image representation

More information

ImageCLEF 2011

ImageCLEF 2011 SZTAKI @ ImageCLEF 2011 Bálint Daróczy joint work with András Benczúr, Róbert Pethes Data Mining and Web Search Group Computer and Automation Research Institute Hungarian Academy of Sciences Training/test

More information

Sketchable Histograms of Oriented Gradients for Object Detection

Sketchable Histograms of Oriented Gradients for Object Detection Sketchable Histograms of Oriented Gradients for Object Detection No Author Given No Institute Given Abstract. In this paper we investigate a new representation approach for visual object recognition. The

More information

Fitting. Lecture 8. Cristian Sminchisescu. Slide credits: K. Grauman, S. Seitz, S. Lazebnik, D. Forsyth, J. Ponce

Fitting. Lecture 8. Cristian Sminchisescu. Slide credits: K. Grauman, S. Seitz, S. Lazebnik, D. Forsyth, J. Ponce Fitting Lecture 8 Cristian Sminchisescu Slide credits: K. Grauman, S. Seitz, S. Lazebnik, D. Forsyth, J. Ponce Fitting We want to associate a model with observed features [Fig from Marszalek & Schmid,

More information

TEXTURE CLASSIFICATION METHODS: A REVIEW

TEXTURE CLASSIFICATION METHODS: A REVIEW TEXTURE CLASSIFICATION METHODS: A REVIEW Ms. Sonal B. Bhandare Prof. Dr. S. M. Kamalapur M.E. Student Associate Professor Deparment of Computer Engineering, Deparment of Computer Engineering, K. K. Wagh

More information

Category vs. instance recognition

Category vs. instance recognition Category vs. instance recognition Category: Find all the people Find all the buildings Often within a single image Often sliding window Instance: Is this face James? Find this specific famous building

More information

Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation

Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation Large-Scale Traffic Sign Recognition based on Local Features and Color Segmentation M. Blauth, E. Kraft, F. Hirschenberger, M. Böhm Fraunhofer Institute for Industrial Mathematics, Fraunhofer-Platz 1,

More information

Multiple Kernel Learning for Emotion Recognition in the Wild

Multiple Kernel Learning for Emotion Recognition in the Wild Multiple Kernel Learning for Emotion Recognition in the Wild Karan Sikka, Karmen Dykstra, Suchitra Sathyanarayana, Gwen Littlewort and Marian S. Bartlett Machine Perception Laboratory UCSD EmotiW Challenge,

More information

Developing Open Source code for Pyramidal Histogram Feature Sets

Developing Open Source code for Pyramidal Histogram Feature Sets Developing Open Source code for Pyramidal Histogram Feature Sets BTech Project Report by Subodh Misra subodhm@iitk.ac.in Y648 Guide: Prof. Amitabha Mukerjee Dept of Computer Science and Engineering IIT

More information

Semantic-based image analysis with the goal of assisting artistic creation

Semantic-based image analysis with the goal of assisting artistic creation Semantic-based image analysis with the goal of assisting artistic creation Pilar Rosado 1, Ferran Reverter 2, Eva Figueras 1, and Miquel Planas 1 1 Fine Arts Faculty, University of Barcelona, Spain, pilarrosado@ub.edu,

More information

Scene Recognition using Bag-of-Words

Scene Recognition using Bag-of-Words Scene Recognition using Bag-of-Words Sarthak Ahuja B.Tech Computer Science Indraprastha Institute of Information Technology Okhla, Delhi 110020 Email: sarthak12088@iiitd.ac.in Anchita Goel B.Tech Computer

More information

Sampling Strategies for Object Classifica6on. Gautam Muralidhar

Sampling Strategies for Object Classifica6on. Gautam Muralidhar Sampling Strategies for Object Classifica6on Gautam Muralidhar Reference papers The Pyramid Match Kernel Grauman and Darrell Approximated Correspondences in High Dimensions Grauman and Darrell Video Google

More information

Object Detection Using Segmented Images

Object Detection Using Segmented Images Object Detection Using Segmented Images Naran Bayanbat Stanford University Palo Alto, CA naranb@stanford.edu Jason Chen Stanford University Palo Alto, CA jasonch@stanford.edu Abstract Object detection

More information

Frequent Inner-Class Approach: A Semi-supervised Learning Technique for One-shot Learning

Frequent Inner-Class Approach: A Semi-supervised Learning Technique for One-shot Learning Frequent Inner-Class Approach: A Semi-supervised Learning Technique for One-shot Learning Izumi Suzuki, Koich Yamada, Muneyuki Unehara Nagaoka University of Technology, 1603-1, Kamitomioka Nagaoka, Niigata

More information

Local Image Features

Local Image Features Local Image Features Ali Borji UWM Many slides from James Hayes, Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Overview of Keypoint Matching 1. Find a set of distinctive key- points A 1 A 2 A 3 B 3

More information

Local invariant features

Local invariant features Local invariant features Tuesday, Oct 28 Kristen Grauman UT-Austin Today Some more Pset 2 results Pset 2 returned, pick up solutions Pset 3 is posted, due 11/11 Local invariant features Detection of interest

More information

Spatial Hierarchy of Textons Distributions for Scene Classification

Spatial Hierarchy of Textons Distributions for Scene Classification Spatial Hierarchy of Textons Distributions for Scene Classification S. Battiato 1, G. M. Farinella 1, G. Gallo 1, and D. Ravì 1 Image Processing Laboratory, University of Catania, IT {battiato, gfarinella,

More information

Object Detection Based on Deep Learning

Object Detection Based on Deep Learning Object Detection Based on Deep Learning Yurii Pashchenko AI Ukraine 2016, Kharkiv, 2016 Image classification (mostly what you ve seen) http://tutorial.caffe.berkeleyvision.org/caffe-cvpr15-detection.pdf

More information

Feature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking

Feature descriptors. Alain Pagani Prof. Didier Stricker. Computer Vision: Object and People Tracking Feature descriptors Alain Pagani Prof. Didier Stricker Computer Vision: Object and People Tracking 1 Overview Previous lectures: Feature extraction Today: Gradiant/edge Points (Kanade-Tomasi + Harris)

More information

Codebook Graph Coding of Descriptors

Codebook Graph Coding of Descriptors Int'l Conf. Par. and Dist. Proc. Tech. and Appl. PDPTA'5 3 Codebook Graph Coding of Descriptors Tetsuya Yoshida and Yuu Yamada Graduate School of Humanities and Science, Nara Women s University, Nara,

More information

CS 231A Computer Vision (Fall 2012) Problem Set 4

CS 231A Computer Vision (Fall 2012) Problem Set 4 CS 231A Computer Vision (Fall 2012) Problem Set 4 Master Set Due: Nov. 29 th, 2012 (23:59pm) 1 Part-based models for Object Recognition (50 points) One approach to object recognition is to use a deformable

More information

2 Related Work. 2.1 Logo Localization

2 Related Work. 2.1 Logo Localization 3rd International Conference on Multimedia Technology(ICMT 2013) Logo Localization and Recognition Based on Spatial Pyramid Matching with Color Proportion Wenya Feng 1, Zhiqian Chen, Yonggan Hou, Long

More information

Dynamic Scene Classification using Spatial and Temporal Cues

Dynamic Scene Classification using Spatial and Temporal Cues 2013 IEEE International Conference on Computer Vision Workshops Dynamic Scene Classification using Spatial and Temporal Cues Arun Balajee Vasudevan, Srikanth Muralidharan, Shiva Pratheek Chintapalli Indian

More information

OBJECT CATEGORIZATION

OBJECT CATEGORIZATION OBJECT CATEGORIZATION Ing. Lorenzo Seidenari e-mail: seidenari@dsi.unifi.it Slides: Ing. Lamberto Ballan November 18th, 2009 What is an Object? Merriam-Webster Definition: Something material that may be

More information

Object recognition (part 1)

Object recognition (part 1) Recognition Object recognition (part 1) CSE P 576 Larry Zitnick (larryz@microsoft.com) The Margaret Thatcher Illusion, by Peter Thompson Readings Szeliski Chapter 14 Recognition What do we mean by object

More information

Supervised learning. y = f(x) function

Supervised learning. y = f(x) function Supervised learning y = f(x) output prediction function Image feature Training: given a training set of labeled examples {(x 1,y 1 ),, (x N,y N )}, estimate the prediction function f by minimizing the

More information

Basic Problem Addressed. The Approach I: Training. Main Idea. The Approach II: Testing. Why a set of vocabularies?

Basic Problem Addressed. The Approach I: Training. Main Idea. The Approach II: Testing. Why a set of vocabularies? Visual Categorization With Bags of Keypoints. ECCV,. G. Csurka, C. Bray, C. Dance, and L. Fan. Shilpa Gulati //7 Basic Problem Addressed Find a method for Generic Visual Categorization Visual Categorization:

More information

Image Classification based on Saliency Driven Nonlinear Diffusion and Multi-scale Information Fusion Ms. Swapna R. Kharche 1, Prof.B.K.

Image Classification based on Saliency Driven Nonlinear Diffusion and Multi-scale Information Fusion Ms. Swapna R. Kharche 1, Prof.B.K. Image Classification based on Saliency Driven Nonlinear Diffusion and Multi-scale Information Fusion Ms. Swapna R. Kharche 1, Prof.B.K.Chaudhari 2 1M.E. student, Department of Computer Engg, VBKCOE, Malkapur

More information

Visuelle Perzeption für Mensch- Maschine Schnittstellen

Visuelle Perzeption für Mensch- Maschine Schnittstellen Visuelle Perzeption für Mensch- Maschine Schnittstellen Vorlesung, WS 2009 Prof. Dr. Rainer Stiefelhagen Dr. Edgar Seemann Institut für Anthropomatik Universität Karlsruhe (TH) http://cvhci.ira.uka.de

More information

Face Recognition Using Vector Quantization Histogram and Support Vector Machine Classifier Rong-sheng LI, Fei-fei LEE *, Yan YAN and Qiu CHEN

Face Recognition Using Vector Quantization Histogram and Support Vector Machine Classifier Rong-sheng LI, Fei-fei LEE *, Yan YAN and Qiu CHEN 2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: 978-1-60595-389-2 Face Recognition Using Vector Quantization Histogram and Support Vector Machine

More information

Tensor Decomposition of Dense SIFT Descriptors in Object Recognition

Tensor Decomposition of Dense SIFT Descriptors in Object Recognition Tensor Decomposition of Dense SIFT Descriptors in Object Recognition Tan Vo 1 and Dat Tran 1 and Wanli Ma 1 1- Faculty of Education, Science, Technology and Mathematics University of Canberra, Australia

More information

Analysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009

Analysis: TextonBoost and Semantic Texton Forests. Daniel Munoz Februrary 9, 2009 Analysis: TextonBoost and Semantic Texton Forests Daniel Munoz 16-721 Februrary 9, 2009 Papers [shotton-eccv-06] J. Shotton, J. Winn, C. Rother, A. Criminisi, TextonBoost: Joint Appearance, Shape and Context

More information

Class 5: Attributes and Semantic Features

Class 5: Attributes and Semantic Features Class 5: Attributes and Semantic Features Rogerio Feris, Feb 21, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Project

More information

Randomized Spatial Partition for Scene Recognition

Randomized Spatial Partition for Scene Recognition Randomized Spatial Partition for Scene Recognition Yuning Jiang, Junsong Yuan, and Gang Yu School of Electrical and Electronics Engineering Nanyang Technological University, Singapore 639798 {ynjiang,jsyuan}@ntu.edu.sg,

More information

Fast Kernel Learning for Spatial Pyramid Matching

Fast Kernel Learning for Spatial Pyramid Matching Fast Kernel Learning for Spatial Pyramid Matching Junfeng He, Shih-Fu Chang Department of Electrical Engineering Columbia University, New York, NY 10027 jh2700,sfchang@columbia.edu Lexing Xie IBM T.J.

More information

Image classification Computer Vision Spring 2018, Lecture 18

Image classification Computer Vision Spring 2018, Lecture 18 Image classification http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 18 Course announcements Homework 5 has been posted and is due on April 6 th. - Dropbox link because course

More information

Summarization of Egocentric Moving Videos for Generating Walking Route Guidance

Summarization of Egocentric Moving Videos for Generating Walking Route Guidance Summarization of Egocentric Moving Videos for Generating Walking Route Guidance Masaya Okamoto and Keiji Yanai Department of Informatics, The University of Electro-Communications 1-5-1 Chofugaoka, Chofu-shi,

More information

The most cited papers in Computer Vision

The most cited papers in Computer Vision COMPUTER VISION, PUBLICATION The most cited papers in Computer Vision In Computer Vision, Paper Talk on February 10, 2012 at 11:10 pm by gooly (Li Yang Ku) Although it s not always the case that a paper

More information

Kernel Codebooks for Scene Categorization

Kernel Codebooks for Scene Categorization Kernel Codebooks for Scene Categorization Jan C. van Gemert, Jan-Mark Geusebroek, Cor J. Veenman, and Arnold W.M. Smeulders Intelligent Systems Lab Amsterdam (ISLA), University of Amsterdam, Kruislaan

More information