Rongrong Ji (Columbia), Yu Gang Jiang (Fudan), June, 2012

Transcription

1 Supervised Hashing with Kernels Wei Liu (Columbia Columbia), Jun Wang (IBM IBM), Rongrong Ji (Columbia), Yu Gang Jiang (Fudan), and Shih Fu Chang (Columbia Columbia) June, 2012

2 Outline Motivations Problem Our Approach Experiments Conclusions CVPR

3 Fast Nearest Neighbor Search Exhaustive search ( time) is inefficient. CVPR

4 Tree Based Indexing O(log n) search time. Impractical for high dimensionality. tree KD tree CVPR

5 Locality Sensitive Hashing [Gionis, Indyk, and Motwani 1999] [Datar et al. 2004] Sublinear search time for approximate NN. Long hash bits (>=1k) and multiple hash tables Query 1 Feature Vector hash function random CVPR

6 Hashing with Compact Codes O(1) search timewithshort bits (<=50) and a single table. Both time and storage efficient. n hash table hash bucket address xi CVPR 2012 q

7 Related Works Three maincategories Unsupervised Hashing LSH, PCAH, ITQ, KLSH, SH, AGH Our Approach Semi Supervised Hashing Supervised Hashing SSH, WeaklySH RBM, BRE, MLH, LDAH CVPR

8 Supervision Semantic Supervision Metric Supervision similar dissimilar dissimilar dissimilar similar CVPR

9 Principle: Preserve Supervised Information The hashing quality could be boosted by leveraging supervised information: similar and dissimilar pairs. dissimilar similar il 0 1 desirable hash hfunction CVPR

10 Outline Motivations Problem Our Approach Experiments Conclusions CVPR

11 Encode Supervised Information Encode as a pairwise label matrix similar pairs dissimilar pairs uncertain The labeled data of samples. Objective: learn r hash functions for r hash bits given and S. CVPR

12 Previous Formulations Goal SSH/OKH [Wang, Kumar, He, Liu, Chang 2010] BRE [Kulis&Darrell 2009] Hamming distance between H(xi) and H(xj) MLH [Norouzi&Fleet 2011] hinge loss CVPR

13 Outline Motivations Problem Our Approach Experiments Conclusions CVPR

14 Proposed Idea: Code Inner Products Optimizing i i Hamming distances can yield ildcompact yet discriminative hash codes, but is hard to implement. We propose to optimize code inner products. code inner product Hamming distance CVPR

15 Hamming Distances The labeled data x1 similar x3 x2 supervised hashinghi Optimization on Hamming distances hash code of x 1 hash code of x distance max distance hash code of x CVPR

16 Code Inner Products The labeled data Optimization i on code inner products x1 similar x2 x 1 code matrix code matrix supervised hashing x Х x 3 fitting x 1 x 2 x 3 r x3 x 1 x 2 pairwise label matrix x x 1 x 2 x 3 S CVPR

17 Code Learning Lead to a clean matrix formed code learning framework ( ): reduce the gap bet. code similarity and semantic similarity sample single hash bit Easy to be extended to a kernelized formulation. CVPR

18 Kernel Based Hash Functions FollowingKLSH KLSH, construct a hash function using a kernel function and m anchor samples: zero mean normalization applied to k(x) kernel matrix =sgn g l samples model parameter m anchors CVPR

19 Sequential Optimization Rewrite the object function as matrix: r bits cumulative vector: kthh bit residue A sequential idea: at a time, only optimize one vector ak provided with the previously optimized one hash bit one time k 1 vectors. CVPR

20 Deal with sgn() We propose two methods to handle sgn(). Spectral Relaxation Generalized SVD Sigmoid Smoothing where is a smooth approximation to sgn(x) ( x >6). Gradient Descent CVPR

21 Outline Motivations Problem Our Approach Experiments Conclusions CVPR

22 CIFAR 10 60K object images from 10 classes, 1K query images. Hamming radius 2 precision in terms of semantic labels. 1K labeled examples are used for (semi )supervised hashing. KSH 0 Spec Relax, KSH Sig Smooth. CVPR

23 Method CIFAR 10 Train Time Test Time 48 bits 48 bits SSH LDAH BRE MLH KSH CVPR 2012 Significant speedup KSH

24 Tiny 1M 1M tiny images from the MIT 80M set, 2K query images. Pseudo labels: top 5% L2 NNs as groundtruths. Hamming radius 2 precision in terms of L2 neighbors. 5K pseudo labeled examples are used for (semi) supervised hashing. CVPR

25 Tiny 1M: Hamming Ranking # returned neighbors # returned neighbors KSH achieves the highest precision and recall. CVPR

26 Tiny 1M: Visual Search Results most visually relevant CVPR

27 Outline Motivations Problem Our Approach Experiments Conclusions CVPR

28 Conclusions A novel inner products based formulation to preserve supervised information into hashing. A sequential code learning procedure: one bit one time. A new smoothing method for binary code optimization. Significant performance gains over state of the arts. Release code soon. S CVPR

29 Thanks! Questions? CVPR