Deep condolence to Professor Mark Everingham

Transcription

1 Deep condolence to Professor Mark Everingham

2 Towards VOC2012 Object Classification Challenge Generalized Hierarchical Matching for Sub-category Aware Object Classification National University of Singapore Learning & Vision Group Shuicheng YAN Jian DONG, Qiang CHEN, Zheng SONG, Yan PAN, Wei XIA Panasonic Singapore Laboratories Media Processing Group Zhongyang HUANG Yang HUA, Shengmei SHEN

3 Framework NUS_PSL_2010 Visual Features Chair Local Feature Extraction Kernel Classification Post Processing Feature Coding VQ Nonlinear Kernel SVM Kernel Regression Feature Pooling SPM Linear Kernel Regression Confidence Refinement with Exclusive prior Detection Results Max pooling

4 Framework NUS_PSL_2012 Subcategory Mining Visual Features Chair Flipping Local Feature Extraction Kernel Classification Post Processing Feature Coding VQ, VQ LLC, FK Nonlinear + Linear Kernel Kernel SVM Kernel Regression Flipping Feature Pooling SPM, SPM GHM Linear Kernel Regression Confidence Refinement with Exclusive prior Flipping Subcategory Detection Results Detection Results Max pooling

5 Framework NUS_PSL_2012 Subcategory Mining Visual Features Chair Ⅱ Subcategory Flipping Mining Flipping Local Feature Extraction Feature Coding VQ, VQ LLC, FK Ⅰ Generalized Hierarchical Matching Feature Pooling SPM, SPM GHM Kernel Nonlinear + Linear Kernel Kernel Linear Kernel Classification SVM Regression Post Processing Kernel Regression Confidence Refinement with Exclusive prior Flipping Subcategory Detection Results Detection Results Max pooling

6 Ⅰ Generalized Hierarchical Matching Traditional Pooling: SPM = approximate geometric constraint Not optimal for object recognition due to misalignment (a) Images (b) SPM partitions (c) Object Confidence Map partition

7 Ⅰ Generalized Hierarchical Matching Encoded local feature vs. side information (a) Side information and Image (b) Hierarchically cluster by side information. Level 1 (top),2 (mid),3 (bottom) (c) Hierarchical structure representation (d) Matching/pooling within each cluster Utilize side information to hierarchically pool local features

8 Ⅰ Generalized Hierarchical Matching Images Side Information - Detection Confidence Map Sliding window sub-window Process Shape Model Score vote back to image Score vote back to image Appearance Model Fusing Object Confidence Maps

9 Ⅱ Sub-Category Mining Intra-class diversity: Foreground distribution is diverse due to appearance, occlusion variance Aspect ratio is not enough to grasp these types of variance

10 Ⅱ Sub-Category Mining Intra-class diversity: Foreground distribution is diverse due to appearance, occlusion variance Aspect ratio is not enough to grasp these types of variance Subcategory awareness is necessary!

11 Ⅱ Sub-Category Mining Inter-class ambiguity: Chairs are ambiguous with sofas

12 Ⅱ Sub-Category Mining Inter-class ambiguity: Some sub-categories may be ambiguous with certain specific object categories Chair Sofa Chair Diningtable

13 Ⅱ Sub-Category Mining Inter-class ambiguity: Some sub-categories may be ambiguous with certain specific object categories Chair Solution: Ambiguity guided subcategory mining Sofa Chair Diningtable

14 Ⅱ Sub-Category Mining Ambiguous Categories Sofa Ambiguity Chair Similarity Subcategory Mining based on both Similarity & Ambiguity Calculate the sample intra-class similarity Calculate the sample inter-class ambiguity Detect dense subgraphs by graph shift algorithm [1] Subgraphs to subcategories. [1] Hairong Liu, Shuicheng Yan. Robust Graph Mode Seeking by Graph Shift. ICML 2010

15 Sub-Category Aware Detection & Classification Chair Training Phase Subcategory 1 Detection Model Contextualized Classification Model Detection Score Testing Phase Visual Feature Subcategory 1 Result Context Feature Subcategory N Result Category Level Result

16 The results Our Best Other's Best Our Best Other's Best Our Best Other's Best aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor MAP

17 Discussions Classification, detection and segmentation are essentially closely related problems. It is predictable that these three problems shall be explored within a unified framework in the near future! Effectiveness seems fine now, how about efficiency?

18 Acknowledgement We would thank Mr. Tsutomu MURAJI, Mr. Keisuke MATSUO, Mr. Ryouichi KAWANISHI from Panasonic Corporation for their support to this collaboration project.

19