Searching in one billion vectors: re-rank with source coding

Transcription

1 Searching in one billion vectors: re-rank with source coding Hervé Jégou INRIA / IRISA Romain Tavenard Univ. Rennes / IRISA Laurent Amsaleg CNRS / IRISA Matthijs Douze INRIA / LJK ICASSP May 2011

2 LARGE media databases, this means Image search 1 million images 2 billion local SIFT descriptors (d=128) Video search: hundred/thousand hours of video E.g., TRECVID evaluation tasks Billions of audio and video descriptors Music retrieval Columbia Million-song database About 1 billion chroma descriptors (d=12, but usually compounded in 60- dimensional vectors to improve time consistency [Serra et al.])

3 Concrete Example: our TRECVID 2010 participation to copy detection Number of indexed vectors (database 400h) Video d= billion image descriptors (SIFT) Audio d= million descriptors (time/frequency energies) Similar videos are retrieved based on these local descriptors For each descriptor, we look at its (Euclidean) nearest neighbors Exhaustive linear search in intractable for search: 1000 descriptors for 1 query frame trillions of high dimensional vector comparisons = in the order of elementary operations Need for powerful approximate neighbors search: CPU and Memory efficient

4 Approximate nearest neighbor (ANN) search Three (contradictory) performance criteria search quality (retrieved vectors are actual nearest neighbors) speed memory usage Most algorithms mainly optimize the two first criteria: LSH, FLANN, etc Locality Sensitive Hashing: good theoretical properties memory consuming: 10 hash tables means at least 40 bytes per vector no data adaptation FLANN [Muja & Lowe 09]: Excellent accuracy/speed trade-off Need a lot of memory: more than 250MB for only 1 million vectors

5 Typical ANN schemes: 2 stages For this second ( re-ranking ) stage, we need raw descriptors, i.e., either huge amount of memory 128GB for 1 billion SIFTs either to perform disk accesses severely impacts efficiency

6 Typical ANN schemes: 2 stages NOT ON A LARGE SCALE For this second ( re-ranking ) stage, we need raw descriptors, i.e., either huge amount of memory 128GB for 1 billion SIFTs either to perform disk accesses severely impacts efficiency In this paper: alternative re-ranking method

7 Searching with quantization [J. et al., TPAMI 11] Main idea: ANN based on a compressed representation of the database vectors Each database vector y is represented by q c (y) where q c (.) is a product quantizer c Search = distance approximation problem The distance between a query vector x and a database vector y is estimated by c Distance is estimated in the compressed domain typically 8 table look-ups and additions per distance estimation (for SIFT) almost as fast as Hamming embedding methods (e.g., Weiss et al.) proved average upper bound on distance approximation error

8 Re-ranking: refine the descriptor s representation A good property of compressed based indexing: explicit (approximate) reconstruction of each database vector Instead of using raw descriptors for the re-ranking stage, refine this reconstruction by encoding the residual vector The residual error vector is This vector error is quantized using another (fine) quantizer q r y is now approximated by which improves the initial estimate Trade-off precision/memory: controlled by a parameter m = number of bytes used to represented an index for quantizer q r

9 Re-ranking: refine the descriptor s representation ALGORITHM Retrieve a list of k potential nearest neighbors using compression-based search method (k >k) d c (x,y) = Perform the explicit reconstruction of the improved estimates c Find the k-nearest neighbors based on improved distance estimates d r (x,y) =

10 Search results: searching in 1 billion vectors Parameters: indexing structure m = 8 bytes per vector m = number of bytes used for the re-ranking stage

11 Efficiency and balancing the bytes The re-ranking stage has a limited cost (almost negligible with typical parameters) Timings: between 1ms and 200ms to search in 1 billion vectors, depending on desired precision For a fixed memory usage, it is more efficient to use less bytes for the first stage and more for the re-ranking stage Moreover, similar or even better search quality:

12 BIGANN: a billion-sized dataset to evaluate approximate search Many ANN algorithms are evaluated on toy datasets Discrepancy of scale between ANN evaluation and real applications Key practical problems are ignored: memory usage not taken into account, over-fitting problems due to many parameters w.r.t. dataset size BIGANN: 128-dimensional SIFT descriptors (David Lowe s implementation) Three subsets 1 billion vectors (the database that we want to search) 10,000 queries 100 million independent vectors to learn the algorithm s parameters Ground-truth is provided for each query Based on exact Euclidean distance comparisons (1 billion x 10,000 in total) Lists of exact 1000 nearest neighbors and corresponding distances

13 Conclusion A source coding based re-ranking approach does not use raw descriptors improved efficiency/quality for fixed memory usage Searching in 1 billion vectors with high precision is possible on a commodity server BIGANN: a large dataset for ANN evaluation Online: toy Matlab package of our compression-based method:

14 Measurement based techniques Several method address this memory problem by designing/learning an embedding function mapping the Euclidean space into the compact Hamming space: Goal: neighborhood in the Hamming space reflects Euclidean neighborhood Related works: Charikar s LSH. More recently Small codes and large databases for recognition, Torralba et al. Spectral Hashing, Weiss et al. 09 But this is exhaustive search. Hybrid approach (non exhaustive+binary codes): Hamming Embedding and Weak geometric consistency for large scale image search, J. et al. 08

15 Searching with quantization [J. et al., TPAMI 11] Vector split into m subvectors: Subvectors are quantized separately by quantizers where each is learned by k-means with a limited number of centroids Example: y = 128-dim vector split in 8 subvectors of dimension components y 1 y 2 y 3 y 4 y 5 y 6 y 7 y centroids q 1 q 2 q 3 q 4 q 5 q 6 q 7 q 8 q 1 (y 1 ) q 2 (y 2 ) q 3 (y 3 ) q 4 (y 4 ) q 5 (y 5 ) q 6 (y 6 ) q 7 (y 7 ) q 8 (y 8 ) 8 bits 64-bit quantization index

16 Product quantizer: asymmetric distance computation (ADC) Compute the square distance approximation in the compressed domain To compute distance between query and many codes compute for each subvector and all possible centroids stored in look-up tables for each database code: sum the elementary square distances Each 8x8=64-bits code requires only m=8 additions per distance! IVFADC: combination with an inverted file to avoid exhaustive search

17 Biased estimator These estimators are biased: bias removed by quantization error terms but does not improve the NN search quality

18 Example Right: 2D anisotropic Gaussian greedy approach selects the dimension that best reduces the distance error per allocated bit Non-integer number of bits per dimension SDC: The number of possible distance is finite, but much higher than for Hamming Embedding up to 2 D instead of D+1 Left: 5 bits cleary more than 6 distances!

19 Performance evaluation with varying R) Comparison with other memory efficient approximate neighbor search techniques Hamming Embedding [Jegou 08] Spectral Hashing [Weiss 09] Performance measured by searching 1M vector varying R) Searching in 1M SIFT descriptors Searching in 1M GIST descriptors

20 Combination with an inverted file

21 Comparison with FLANN [Muja & Lowe 09] Tested on 1 million SIFTs 1.5 to 2 faster than FLANN for same accuracy Memory usage for 1M vectors (according to top command): FLANN: > 250MB Ours: < 25MB