MULTI-INDEX VOTING FOR ASYMMETRIC DISTANCE COMPUTATION IN A LARGE-SCALE BINARY CODES. Chih-Yi Chiu, Yu-Cyuan Liou, and Sheng-Hao Chou

Transcription

1 MULTI-INDEX VOTING FOR ASYMMETRIC DISTANCE COMPUTATION IN A LARGE-SCALE BINARY CODES Chih-Yi Chiu, Yu-Cyuan Liou, and Sheng-Hao Chou Departent of Coputer Science and Inforation Engineering, National Chiayi University, Taiwan ABSTRACT In this paper, we propose a fast search ethod for asyetric distance coputation in large-scale binary codes. Asyetric distances take advantage of less inforation loss at the query side. However, calculating asyetric distances with the linear scan approach is prohibitive in a large-scale dataset. We present a novel algorith called ulti-index voting, which integrates the ulti-index hashing technique with a voting echanis, to select appropriate candidates and calculate their asyetric distances. Substantial experiental evaluations are given to deonstrate that, guided by the voting echanis, the proposed ethod can yield an approxiate accuracy to the linear scan approach while accelerating the run tie with a significant speedup. For exaple, one result shows that in a dataset of one billion 256-bit binary codes, exaining only.5% of the dataset can reach 95~99% close accuracy to the linear scan approach but can accelerate over 73~128 ties. Index Ters Nearest neighbor search, asyetric distance coputation, ulti-index hashing and voting 1. INTRODUCTION Searching the nearest neighbors (NN) is a proble of finding the closest data with respect to a given query. It is a fundaental requireent in any applications, such as inforation retrieval, signal processing, pattern recognition, and coputer vision. However, within a high-diensional feature space and a large-scale dataset volue, the NN search becoes a very challenging task when faced with the accuracy, efficiency, and eory issues siultaneously. A nuber of binary ebedding algoriths have been recently proposed to address these issues [1][3][4][7][8][1]. By ebedding the original data into the binary space, the binary ebedding algoriths enable a faster NN search by hardware-supported bit operations and anageable eory consuption with the copact binary representation. However, the use of binary ebedding causes severe inforation loss after ebedding. Even though these binary ebedding algoriths try to bridge the gap through learning, an inevitable large inforation loss exists in apping a realvalued vector to a binary code. Besides, the Haing distance between two binary codes allows only a few distinct values. Hence, the NN search in the binary space causes a weaker discriination than that in the real-valued space. A novel concept called asyetric distance atching for binary codes has recently been proposed, and has been shown to be ore accurate than Haing distance atching [2][5]. Asyetric eans that the distance is coputed between two different spaces. For exaple, in our case, the query is a real-valued vector while the reference data are binary codes. Since asyetric atching does not ebed the query to the binary space, it takes advantage of a lesser aount of inforation loss at the query side. The NN search in the real-valued space would yield a ore discriinating result. We follow Gordo et al. s forulation of asyetric distances for binary ebedding codes [2]. Assue that a binary ebedding function h(x i ) = q(f(x i )) can be decoposed into two functions f and q, where f(x i ): {R} s {I} t transfors s-diensional real-valued vector x i to a t-diensional real-valued vector y i, and q(y i ): {I} t {B} t transfors y i to a t-diensional binary code z i. 1 The asyetric distance between a real-valued vector (query) and a binary code (reference) is calculated in the interediate space {I} t. In Gordo et al. s approach, they adopted the linear scan schee to exhaustively calculate the asyetric distances of all reference data. However, this is prohibitive when searching in a large-scale dataset. In this paper, we propose a fast search ethod for asyetric distance coputation in large-scale binary codes. For a reference dataset of binary codes, we divide each code into disjoint binary subvectors and store the in different index tables. Given a query of the real-valued vector, it is transfored and divided into interediate real-valued subvectors. We calculate the Euclidean distances in the interediate space between each query subvector and centroids of all entries in an index table, and rank these entries fro near to far. We then sequentially scan the ranked entries across the index tables for voting. A reference binary code that has ultiple subvectors close to the query, i.e., it gets enough votes fro the query, is selected as a candidate. Finally, we calculate candidates asyetric distances and output NNs for the query. 1 R, I, and B represent the original real-valued, interediate, and ebedded binary spaces, respectively /16/$ IEEE 116 ICASSP 216

2 The candidate selection algorith is called ulti-index voting. Epirical studies are given through substantial experiental evaluations on large-scale datasets. Results show that, guided by the voting echanis, the proposed ethod deonstrates a satisfactory accuracy with a significant speedup. For exaple, in a dataset of one billion 256-bit binary codes, our ethod that exaines only.5% of the reference dataset can reach 95~99% close accuracy to the linear scan approach but can accelerate over 73~128 ties [2]. Two state-of-the-art ethods for asyetric NN search, including product quantization [5] and ulti-index hashing [9] are ipleented for coparison. The proposed ethod yields the best accuracy by taking advantage of cobining ulti-index voting and interediate space coputation. In particular, without the voting echanis, the accuracy is uch lower than the linear scan approach; adopting the voting echanis can generate a burst iproveent in accuracy. The reainder of this paper is organized as follows. In Section 2, we briefly introduce the related work. Section 3 describes and analyzes the proposed ulti-index voting algorith in detail. Section 4 deonstrates and discusses the experiental results. Conclusions are given in Section RELATED WORK The NN search for binary codes can be divided into two categories: syetric and asyetric. For the syetric approach, a query x Q is transfored to a binary code through the binary ebedding function. The distance fro x Q to a reference data x i is expressed as the Haing distance of their binary codes. Matching binary codes can be done efficiently with a couple of achine instructions (e.g., XOR and POPCNT instructions). In particular, Norouzi et al. [9] proposed an efficient ulti-index hashing (MIH) schee to perfor an exact search for binary codes. Reference binary codes are divided into disjoint subvectors and hashed into different index tables. Given a query and a search radius r, the reference codes that differ fro the query by r bits or less can be exactly returned. A key factor for speedup is that for each index table, the search radius is decreased to only r bits. This is because, when two binary codes differ by r bits or less, at least in one of their subvectors they differ by at ost r bits. Rather than the syetric approach that purely exploits the binary space structure, the asyetric approach utilizes the original space inforation of binary codes. However, the asyetric distance between x Q and x i is coputed in neither the original space nor the binary space; it is calculated in their interediate space. Gordo et al. [2] presented two asyetric distance coputation (ADC) schees based on statistic expectation and lower-bound. Jégou et al. [5] proposed an inverted file syste with asyetric distance coputation (IVF-ADC) for approxiate NN search based on product quantization. The inverted file syste is served as a coarse quantizer. The residual vector between a vector and its corresponding coarse centroid is encoded with a product quantizer, which proceeds by decoposing the high-diensional space into a Cartesian product of low-diensional subspaces. 3. MULTI-INDEX VOTING Suppose we have a reference dataset of n t-diensional binary codes {z i {B} t i = 1, 2,, n}, whose correspondences are s-diensional real-valued vectors in the original space {x i {R} s i = 1, 2,, n} and t- diensional real-valued vectors in the interediate space {y i {I} t i = 1, 2,, n}. Every binary code z i is divided into disjoint subvectors, each of which coprises of g = t bits (assue t is divisible by ). g should be chosen appropriately so that the syste can bear O(2 g ) data space occupied in eory. We then build index tables with respect to subvectors as follows. Denote the jth subvector as z i, j [1, ]. A cluster Z β = {i z i β} is generated to store a set of reference identities whose jth subvectors are, where β {B} g is a g-diensional binary code. We also copute the cluster ean in the interediate space of Z β : y β = 1 Z β y i i Z β where denotes the cardinality of the set. For any subvector z i = β, its correspondence in the interediate space can be considered to be y β. In other words, we build a quantizer u of the jth subvector in the interediate space: u (y i ) = y β for i Z β. Z β and y β are calculated for all j and offline. An online knn search algorith for the given query x Q {R} s is presented hereafter. The query x Q is projected to the interediate space as y Q and then divided into subvectors. For the jth subvector y Q, we sort the clusters {Z β } based on its Euclidean distances to the cluster eans {y β } in ascending order. Denote the sorted cluster indexes as {B r r = 1,2,, 2 g }, where y Br is the rth nearest to y Q in the interediate space. Started fro the nearest clusters of all subvectors {Z B1 j = 1,2,, }, if a reference data x i is stored in Z B1, we denote x i is voted by x Q. When the vote count of x i accuulates ore than the vote threshold T v, x i is put in a candidate set. The process is repeated for the next nearest clusters until the size of the candidate set is greater than the candidate threshold T c. The search algorith, tered ulti-index voting, is listed in the following:, 1161

3 Algorith: Multi-index voting Input: n reference binary codes {z i }, index tables {Z β }, and the query x Q. Output: k NN binary codes for x Q. 1. Generate a t-diensional vector in the interediate space for x Q : y Q = f(x Q ). 2. For each subvector index j and binary code, calculate the Euclidean distance between y Q and yβ : ed (y Q, yβ ) = yq yβ For each j, sort the above Euclidean distances to obtain an index list {B r r = 1,2,, 2 g }, where y Br is the rth nearest to y Q in the interediate space. 4. Initialize an epty candidate set C and vote counts for all x i : x i.votecount =, i = 1, 2,, n. 5. Let C be the size of C, T v and T c be the thresholds of the vote count and the candidate size respectively. Do the loop: do for r = 1 to 2 g for j = 1 to for i Z Br x i.votecount = x i.votecount + 1. if x i.votecount T v Add x i to C. while C < T c. 6. For each x i C, calculate the asyetric distance between x Q and x i in the interediate space: ad(x Q, x i ) = ed (y Q, yβ ), j=1 where i Z β. Build a axheap to store the k sallest asyetric distances seen so far. 7. Apply the heap sort algorith to the axheap and output the k NN binary codes. 4. EXPERIMENTAL RESULT We carried out the experients on a public large-scale dataset: ANN_SIFT1B [6]. It contains one billion SIFT vectors (each with 128-diensional real values) and 1, queries along with the ground truth of 1, nearest neighbors for each query. The real-valued vectors are transfored to 128/256 bits binary codes by using locality sensitive hashing (LSH). Experients were run on a PC using Windows 7, with Intel Core i7 3.4GHz CPU (only one thread is used), 256KB L2 cache, and 64GB RAM. We ipleent five NN search ethods for coparison, including exhaustive syetric distance coputation (abbreviated as SDC), exhaustive asyetric distance coputation (ADC) [2], inverted file syste with asyetric distance coputation (IVF-ADC) [5], ultiindex hashing with asyetric distance coputation (MIH- ADC) [9], and the proposed ulti-index voting with asyetric distance coputation (MIV-ADC). The forer two ethods perfor an exact but exhaustive search. They can be considered the baselines of the syetric and asyetric approaches, respectively. The latter three ethods act as the approxiate but speedup versions of ADC. The ain difference aong the is in the inverted indexing: IVF-ADC eploys a single index table; MIH- ADC and MIV-ADC adopt ultiple index tables but with different candidate selection echaniss. MIH-ADC selects candidates that are hit by the query only once and within r bits Haing distance to the query in the binary space. MIV-ADC applies a voting echanis to select candidates if they are hit by the query at least T v ties in different index tables, and candidates are selected in order according to their Euclidean distances to the query in the interediate space. Figure 1 plots ean average precision () for SDC, ADC, and MIV-ADC. The abscissa axes represent T v, and the ordinate axes represent. Solid curves represent various T c = {1K, 5K, 1K, 5K, 1M} of MIV-ADC, while dash lines represent SDC and ADC. In general, ADC stands for the accuracy upper bound for in asyetric binary code search. MIV-ADC gets an iproveent on accuracy with an increasing voting threshold T v. However, it turns out to be degenerated when T v is close to the nuber of index tables due to the cuulative quantization error in the interediate space MIV-ADC_1K MIV-ADC_5K MIV-ADC_1K MIV-ADC_5K MIV-ADC_1M ADC SDC T v T v (a) (b) Fig. 1. for querying the 1B (a) 128-bit and (b) 256-bit datasets. Solid curves represent various T c = {1K, 5K, 1K, 5K, 1M} of MIV-ADC, while dash lines represent SDC and ADC. Tables I and II suarize the run tie for the 128-bit and 256-bit datasets, respectively. In general, as the candidate size T c increases ten ties, the run tie increases uch less than ten ties. It iplies the proposed ethod runs in a sublinear tie coplexity. In addition, the speedup of MIV-ADC over ADC is ore proinent in a longer-bit dataset. As an exaple of the 256-bit dataset, when T v = 3 and T c = 5K (.5% of the reference dataset), MIV-ADC can reach 99% close to ADC but run over 73 ties faster than ADC; if we change T v = 2, MIV-ADC can reach 95% close accuracy to ADC with 128 ties acceleration over ADC MIV-ADC_1K MIV-ADC_5K MIV-ADC_1K MIV-ADC_5K MIV-ADC_1M ADC SDC 1162

4 Table I. Run tie (second) for querying the 1B 128-bit dataset T v = 1 T v = 2 T v = 3 T v = 4 T v = 5 T v = 6 T v = 7 T v = 8 T c = 1K T c = 5K MIV-ADC T c = 1K T c = 5K T c = 1M E-SDC E-ADC Table II. Run tie (second) for querying the 1B 256-bit dataset T v = 1 T v = 2 T v = 3 T v = 4 T v = 5 T v = 6 T v = 7 T v = 8 T c = 1K T c = 5K MIV-ADC T c = 1K T c = 5K T c = 1M E-SDC E-ADC Next, we copare and discuss the three approxiate ADC ethods: IVF-ADC, MIH-ADC, and MIV-ADC. Figure 2 shows and run tie for querying the 128-bit dataset. All ethods apply the sae feature decoposition schee by partitioning the original SIFT vector into eight disjoint subvectors. In the 128-bit dataset, IVF-ADC eploys one index table, whereas MIH-ADC and MIV- ADC eploy eight index tables; each index table coprises of 2 16 entries (clusters). We do not show the result of the 256-bit dataset because MIH-ADC requires ore eory beyond the 64GB RAM achine. Despite the voting echanis, MIV-ADC yields the best accuracy aong these ethods. Considering MIV- ADC and MIH-ADC, they both use ultiple index tables but apply different candidate selection strategies. MIV-ADC selects candidates in the interediate space, whereas MIH- ADC does it in the binary space, which is known to be less discriinatory. Therefore, MIV-ADC can obtain higher quality candidates to yield a better accuracy than MIH-ADC. An interesting observation is that, in Figure 2(b), the run tie of MIH-ADC and MIV-ADC has a crossover in the interval T c = {1K, 5K}. In other words, when T c is raised, MIH-ADC will take ore tie to collect ore candidates than MIV-ADC. This is because the candidate set of MIH-ADC grows rapidly with the order of the binoial coefficient, whereas that of MIV-ADC increases gradually in the proposed voting algorith. For the run tie, IVF-ADC is the fastest NN search ethod due to its siple single-index structure. Without the voting echanis, however, the accuracy is uch lower than the accuracy upper bound posed by the linear scan ethod ADC. It requires a large aount of candidates to be exained to approach the accuracy upper bound, and thus uch ore tie is spent for the candidate exaination. MIV-ADC that adopts the voting echanis can generate a burst iproveent in accuracy, which is close to the accuracy upper bound, by exaining relatively sall aount of candidates. For exaple in the 256-bit dataset, IVF-ADC yields.15 by spends 5.8 seconds to scanning one hundred illion candidates (1% of the 1B reference dataset), while MIV-ADC yields the sae accuracy by spending.93 seconds to scan five hundred thousand candidates (.5% of the 1B reference dataset) under T v = IVF-ADC MIH-ADC MIV-ADC T =1 v =2 =3 1K 1K 5K 1M T c (a) (b) Fig. 2. (a) and (b) run tie for searching the 1B 128- bit SIFT dataset by IVF-ADC, MIH-ADC, and MIV-ADC. 5. CONCLUSIONS In this paper, we present an approxiate asyetric NN search ethod for binary ebedding codes. By taking advantage of ulti-index voting in the interediate space, the proposed ethod achieves an outstanding perforance in ters of the tradeoff between the accuracy and the efficiency. Experiental results show the proposed NN search ethod yields a close accuracy to the linear scan approach and a significant speedup by applying a sall vote threshold in exaining a fraction of the dataset. ACKNOWLEDGEMENTS 1K 1K 5K 1M T c This work was supported in part by the Ministry of Science and Technology, Taiwan, under Grants MOST E MY3 and MOST E MY2. run tie IVF-ADC MIH-ADC MIV-ADC T =1 v =2 =3 1163

5 REFERENCES [1] T. Ge, K. He, and J. Sun, Graph cuts for supervised binary coding, In Proceedings of European conference on Coputer Vision (Zurich, Switzerland, Sep. 6-12, 214). ECCV 14. [2] A. Gordo, F. Perronnin, Y. Gong, and S. Lazebnik, Asyetric distances for binary ebeddings, IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 36, No. 1, pp , 214. [3] K. He, F. Wen, and J. Sun, K-eans hashing: an affinity-preserving quantization ethod for learning binary copact codes, In Proceedings of IEEE International Conference on Coputer Vision and Pattern Recognition. (Portland, USA, Jun ). CVPR 13. [4] G. Irie, Z. Li, X. M. Wu, and S. F. Chang, Locally linear hashing for extracting non-linear anifolds, In Proceedings of IEEE International Conference on Coputer Vision and Pattern Recognition (Colubus, USA, Jun , 214). CVPR 14. [5] H. Jégou, M. Douze, and C. Schid, Product quantization for nearest neighbor search, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 1, pp , 211. [6] H. Jégou, R. Tavenard, M. Douze, and L. Asaleg, Searching in one billion vectors: re-rank with source coding, In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (Prague, Czech Republic, May 22-27, 211). ICASSP 11. [7] W. Liu, J. Wang, S. Kuar, and S. F. Chang, Hashing with graphs, In Proceedings of International Conference on Machine Learning (Bellevue, USA, Jun. 28-Jul. 2, 211). ICML 11. [8] M. Norouzi and D. J. Fleet, Minial loss hashing for copact binary codes, In Proceedings of International Conference on Machine Learning (Bellevue, USA, Jun. 28-Jul. 2, 211). ICML 11. [9] M. Norouzi, A. Punjani, and D. J. Fleet, Fast exact search in Haing space with ulti-index hashing, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 6, pp , 214. [1] Y. Weiss, A. Torralba, and R. Fergus, Spectral hashing, In Proceedings of Annual Conference on Neural Inforation Processing Systes (Vancouver, Canada, Dec. 8-13, 28). NIPS