An Latent Feature Model for

Transcription

1 An Latent Feature Model for Mario Fritz UC Berkeley Michael Black Brown University Gary Bradski Willow Garage Sergey Karayev UC Berkeley Trevor Darrell UC Berkeley

2 Transparent objects are ubiquitous in environments Relevant to service robots local feature approach inappropriate Full physical model intractable

3 Related Work Material Finding Glass Specular Surfaces in Natural Images Classifying Materials from their Reflectance Low- Level Image Cues in the of Translucent Materials [Fleming, on Applied 05] by Using for Transparent and Layered Phenomena Eg [Ben- and rendering of pacerns Environment Madng and

4 Related Work Material recogni-on: Finding Glass Detec-ng Specular Surfaces in Natural Images Classifying Materials from their Reflectance Proper-es Low- Level Image Cues in the Percep-on of Translucent Materials [Fleming, Transac-ons on Applied Percep-on 05] by Using for Transparent and Layered Phenomena Eg [Ben- and rendering of pacerns Environment Madng and

5 Related Work Material Finding Glass Specular Surfaces in Natural Images Classifying Materials from their Reflectance Low- Level Image Cues in the of Translucent Materials [Fleming, on Applied 05] Recogni-on by Speculari-es: Using Speculari-es for Recogni-on Transparent and Layered Phenomena Eg [Ben- and rendering of pacerns Environment Madng and

6 Related Work Material Finding Glass Specular Surfaces in Natural Images Classifying Materials from their Reflectance Low- Level Image Cues in the of Translucent Materials [Fleming, on Applied 05] by Using for Transparent Mo-on and Layered Phenomena Eg [Ben- Acquisi-on and rendering of refrac-ve pa_erns Environment Ma`ng and Composi-on

7 Related Work Material Finding Glass Specular Surfaces in Natural Images Classifying Materials from their Reflectance Low- Level Image Cues in the of Translucent Materials [Fleming, on Applied 05] by Using for Transparent and Layered Phenomena Eg [Ben- and rendering of pacerns Environment Madng and Non of these approaches addresses transparent objects recogni-on in real- world condi-ons

8 Local Feature- based x 1 Codebook: clustercenter 1 clustercenter N y 1 x 1 y 1 f( I 1 ) x N y N f( I N ) quan@ze ˆx 1 ŷ 1 clusterid 1 ˆ x N ˆ y N clusterid N Codebook clusters assume prototypical global patch appearance histogram d classifier

9 SIFT- type Descriptors I f(i) SIFT is popular choice for local feature computa@on It performs spa@al binning of orienta@on quan@zed gradient informa@on Unnormalized distribu@on over local gradient sta@s@cs We will use the a par@cular visualiza@on as proposed for the related HOG method f

10 The Problem of Transparency Significant in patch appearance Ogen gradient energy is dominated by background

11 The Problem of Transparency Significant in patch appearance Ogen gradient energy is dominated by background but common latent structure

12 The Problem of Transparency x 1 Codebook: clustercenter 1 clustercenter N y 1 x 1 y 1 f( I 1 ) x N y N f( I N ) quan@ze ˆx 1 ŷ 1 clusterid 1 ˆ x N ˆ y N clusterid N histogram Codebook clusters assume prototypical global patch appearance d classifier

13 The Problem of Transparency x 1 Codebook: clustercenter 1 clustercenter N y 1 x 1 y 1 f( I 1 ) x N y N f( I N ) quan@ze ˆx 1 ŷ 1 clusterid 1 ˆ x N ˆ y N clusterid N histogram Codebook clusters assume prototypical global patch appearance d classifier

14 Key Idea: Local Latent x 1 Components: component 1 component T y 1 x 1 y 1 f( I 1 ) x N y N f( I N ) Latent component latent model ˆx 1 ŷ 1 componentids 1 ˆ x N ˆ y N componentids N histogram Codebook is replaced by a set of latent components d classifier

15 Local Feature Model Factor gradient descriptor into Unknown non- mixture weights θ (1) φ (1) Unknown mixture components Regularize with sparsity Advantages vs eg VQ, PCA: θ (2) (T 1) θ φ (2) (T 1) φ model allows for superimposed structures Appropriate model for factorizing local gradient No reliance on global patch appearance θ (T ) φ (i) φ (T )

16 LDA- SIFT Factor SIFT descriptor into latent components using LDA/sLDA [Blei03,Griffiths04,Blei07]: is realized as mixture model sparsity is implemented as Dirichlet priors Graphical model Document = Patch Dirichlet prior word 1 word 144 Transparent

17 LDA- SIFT Factor SIFT descriptor into latent components using LDA/sLDA [Blei03,Griffiths04,Blei07]: is realized as mixture model sparsity is implemented as Dirichlet priors Graphical model Learnt mixture components Document = Patch Dirichlet prior word 1 word 144 Transparent

18 Comparison to previous SIFT/LDA

19 Transparent Visual Words Latent component Average occurrence on train Occurrences on test

20 Architecture Infer transparent visual words glass T LDA X Y Classifier background T X Y

21 Experiments

22 Data

23 Results vs baseline Training on 4 different glasses in front of screen Tes@ng on 49 glass instances in home environment Sliding window linear SVM- detec@on

24 Architecture glass T LDA X Y Classifier background T X Y

25 Results: general vocabulary Training on 4 different glasses in front of screen Tes@ng on 49 glass instances in home environment Sliding window linear SVM- detec@on

26 Architecture glass T slda X Y Classifier background T X Y

27 Results: slda Training on 4 different glasses in front of screen Tes@ng on 49 glass instances in home environment Sliding window linear SVM- detec@on

28 Conclusion local feature models (VQ, NN) are poorly suited for transparent object Proposed local feature models can detect superimposed image structures Developed approach to learn such using topic models Sparse of local gradient Encouraging results on real- world data

29 Future Work Different feature extend model in hierarchical fashion of material property cues; inverse local light transport models Explore benefits for opaque object understand to sparse image coding as well as to biological models

30 Thank you for your