What Is the Most Efficient Way to Select Nearest Neighbor Candidates for Fast Approximate Nearest Neighbor Search?

Transcription

1 IEEE Internatonal Conference on Computer Vson What Is the Most Effcent Way to Select Nearest Neghbor Canddates for Fast Approxmate Nearest Neghbor Search? Masakazu Iwamura, Tomokazu Sato and Koch Kse Graduate School of Engneerng, Osaka Prefecture Unversty {masa, Abstract Approxmate nearest neghbor search (ANNS) s a basc and mportant technque used n many tasks such as object recognton. It nvolves two processes: selectng nearest neghbor canddates and performng a brute-force search of these canddates. Only the former though has scope for mprovement. In most exstng methods, t approxmates the space by quantzaton. It then calculates all the dstances between the query and all the quantzed values (e.g., clusters or bt sequences), and selects a fxed number of canddates close to the query. The performance of the method s evaluated based on accuracy as a functon of the number of canddates. Ths evaluaton seems ratonal but poses a serous problem; t gnores the computatonal cost of the process of selecton. In ths paper, we propose a new ANNS method that takes nto account costs n the selecton process. Whereas exstng methods employ computatonally expensve technques such as comparatve sort and heap, the proposed method does not. Ths realzes a sgnfcantly more effcent search. We have succeeded n reducng computaton tmes by one-thrd compared wth the state-of-theart on an experment usng mllon SIFT features.. Introducton Fndng the nearest neghbor (NN) of a gven query, called nearest neghbor search (NNS), s a smple but mportant task. An approxmate nearest neghbor search (ANNS) s an NNS n whch an approxmaton s ntroduced. ANNS problems are actvely studed and wdely appled to many applcatons such as near-duplcate detecton [], largescale object recognton [,,,, ], document mage retreval [], camera-based character recognton [, ]. The requred features for ANNS methods are accuracy n fndng the true NNs, and effcency n tme and space. In ths paper, we focus on the relatonshp between accuracy and computatonal effcency. Takng nto account the relatonshp between accuracy and computaton tmes s nherent n NNS problems. If the computaton tme s not an ssue, NNS problem can always be solved by a brute-force search,.e., calculatng all the dstances between the query and the data. Ths nave soluton s not practcal because t can take a long tme, especally for large datasets. A way to reduce computaton tmes s to reduce the data submtted to the brute-force search. That s, a lmted number of data, where we call NN canddates, s selected n the selecton process followed by the brute-force search. Only n the selecton process s there occason to ntroduce mprovements n the NNS problem and t needs to be fast and accurate. Lengthy computaton tmes n selectng the NN canddates negates the advantage n reducng the data. By not beng accurate n selectng NN canddates, the number of NN canddates needs augmentng to compensate for the low accuracy. Computaton tmes though would lengthen n the brute-force searches. The performance of most exstng methods s evaluated based on accuracy as a functon of the number of canddates. The number of canddates determnes computaton tmes for brute-force searches. However, t s ndependent of the computaton tmes of the selecton process. Hence papers gnorng the computaton tme of the selecton process tackle only a part of the ANNS problem. Some papers take nto account computaton cost properly (e.g., [,, ]). Among them, the most effcent method s the nverted mult-ndex () []. However, on the queston, What s the most effcent way to select NN canddates?, we found that the method does not capture the essence of the NN canddate selecton. An ntutve noton of what we mean s the followng. Imagne that there are n data and a task to fnd NN canddates of sze k (k <n). The worst approach s sortng them all, a task whch takes tme O(n log n). A better way s to use a partal sort technque usng a prorty queue. That s, each datum s added to the prorty queue and the best k data are kept. Ths s One mght beleve that computaton tmes for the selecton process can be gnored n comparson wth that for the brute-force search. However, a good ANNS method reduces the computaton tmes for brute-force searches. As a result, these computaton tmes are not the domnant factor. -/ $. IEEE DOI./ICCV..

2 known to take tme O(n log k). takes ths approach. However, there s a faster sort algorthm the bucket sort. It can complete the task n tme O(n) because t does not perform comparsons between data. Thus f we can prepare approprate buckets, the selecton of NN canddates can be much faster. Ths makes sense because our purpose s just fndng NN canddates; there s no need to arrange buckets n the ascendng order of ther dstances. Towards a practcally fast ANNS method for large scale datasets, we propose a more effcent ANNS method compared wth others, whch nvolves the most effcent way of fndng NN canddates. The proposed method s named bucket dstance hashng (). In the experments, we compared the proposed method wth varous representatve ANNS methods on the same platform wth respect to the crteron, recall as a functon of computaton tme, n addton to the commonly used crteron, recall as a functon of the number of canddates. Although we focus on the NN search, the same technque s drectly applcable to the k- NN search problem.. Related Work For an effcent selecton of NN canddates, data are usually ndexed n advance. Based on the structure for ndexng, ANN methods can be dvded nto tree-based and hashbased. Hash-based methods are also classed ether as data ndependent or data dependent. The former does not use data for ndexng though the latter does... Tree-based approach FLANN s a representatve tree-based method []. It automatcally selects the best method among the randomzed kd-tree [], the herarchcal k-means [] and the brute-force search, and tunes parameters for a gven dataset. In the experments, randomzed kd-tree and herarchcal k-means were compared wth the proposed method as FLANN tself dd not work for large datasets... Hash-based data ndependent approach Localty senstve hashng (LSH) s a representatve hash-based data ndependent method [, ]. There are many mprovements (e.g., [, ]). However, ts performance s known to be worse than the data dependent ndexng methods. We also confrmed that LSH s far slower than the data dependent methods that follow... Hash-based data dependent approach The data dependent methods can be categorzed nto those performng the selecton of NN canddates n the Eucldean space and the Hammng space. Selecton n the Eucldean space In ths category, the methods use quantzaton and data compresson technques. Vector quantzaton (VQ) s used n some ANNS methods such as VQ-ndex [7] and []. Product quantzaton (PQ) s used n []. As mentoned above, outperforms. Transform codng s used n []. The process can be regarded as equvalent to scalar quantzaton (SQ). From the category, we selected and for the evaluaton; both are revewed n the next secton. Although we mplemented transform codng, t was not comparable to those two. Selecton n the Hammng space In ths category, data are represented by bnary codes. Ther use helps reduce memory usage and computaton tmes n Hammngdstance calculatons because of the effcent btwse XOR operaton. Because of these good propertes, many ANNS methods based on bnary codes have been proposed (e.g., [,, 7,,, 7,, ]). Among them, spectral hashng (SH) [] s a representatve method. To acheve hgh recall, long codes, of length or bts long, are often used. Actually handlng such a long code s not easy. A baselne method for fndng NN canddates n Hammng space s the lnear search whch, whle fast, s ntractable when appled to large datasets. One mght thnk that the tree or hash structure helps to acheve sublnear searches. However, the number of data havng the same Hammng dstance to the query explode as the dstance ncreases, whch prevents the NN canddate selecton process from beng effcent []. As ponted out n [], one dea s to use the orgnal LSH for bts []. However, the data ndependent approach s neffcent as mentoned above. Another dea mght be to apply more sophstcated approaches used n exstng ANNS methods desgned for Eucldean space. We appled the proposed selecton method of NN canddates to SH. However, the performance dd not compare well wth other representatve methods.. Exstng Hash-based Indexng Methods Usng Vector and Product Quantzaton Revewng and n ths order helps understand the proposed method. Thus n ths secton, we present them n greater detal than Sec.... [] ndexes data usng VQ n selectng NN canddates effcently. Data are dvded nto clusters usng the k-means clusterng algorthm. Gven a query, the clusters close to the query are searched and data belongng to the clusters are selected as NN canddates. There s a possblty that some data close to the query are not ncluded n the NN canddates. However, VQ s known to be the best quantzaton method n respect to quantzaton error for the

3 same number of clusters []. Thus, t s expected to maxmze the accuracy of NN canddate selecton... Inverted mult-ndex () As a better soluton than, Babenko and Lemptsky proposed [], whch uses PQ nstead of VQ. PQ s an ntermedate quantzaton method between SQ and VQ; a vector space s dvded nto subspaces and then VQ s appled to the subvectors n each subspace. PQ generally produces larger errors than VQ wth the same number of clusters. Nevertheless, computaton tmes to acheve the same recall can be reduced wth the mult-sequence algorthm (MSA). Fgure llustrates an overvew of. Referrng to the fgure, there are four clusters n the subspace and three clusters n the subspace. The centrods of the clusters are represented by Cj, where labels each subspace and j labels a cluster n the -th subspace. Intally, the squared dstances between the query denoted by a star and the centrods n the subspaces are calculated. The squared dstance n the orgnal space s gven by the sum of the squared dstances n the subspaces. Then, f all the dstances n the orgnal space are calculated, the centrods close to the query n the orgnal space are found. However, t s not necessary to calculate all the dstances n the orgnal space, because centrods n subspaces wth large dstances cannot contrbute to the process. Hence, they are gnored n MSA. Fgure provdes an overvew of MSA. In MSA, the squared dstances n each subspace are sorted frst. Then, the centrods n the subspaces are examned n ascendng order of dstance. As shown n Fg. (a), the frst cluster examned s the drect product C C whose squared dstance s. The neghborng clusters C C and C C are selected as canddates to be examned n the next turn. Fgure (b) shows the next step when the canddate cluster C C whose squared dstance s s selected because ts squared dstance s the smallest among the canddates. Then, the cluster C C whose squared dstance s s selected as a new canddate. The reason that the cluster C C (wth squared dstance of but hdden n the fgure) s not selected as a canddate s that at least one canddate s guaranteed to have a smaller dstance than the cluster (see [] for more detal). Ths process lasts untl a suffcent number of NN canddates s obtaned. As seen above, MSA compares the dstances between clusters and arranges them n ascendng order of dstance. Ths s not requred n selectng the NN canddates. In ths regard, the method does not capture the essence of NN canddate selecton. The fgures and llustrate the case when the feature space s dvded nto two. If t s dvded nto more than two, the computatonal cost expands. Thus t s reported that performed best when the feature space s dvded nto two. We shall show that space dvson nto Squared dstances between the query and centrods n the orgnal subspace Centrods n subspace C C C C Squared dstances between the query and centrods n each subspace x Centrods n the orgnal space x C C C Centrods n subspace Query Fgure : Overvew of the nverted mult-ndex () when the feature space s dvded nto two. C C C C C C C C C C C C C C (a) C C C (d) C C C C C C C C (b) C C C (e) C C C C C C C C C C C (c) C C C Fgure : Overvew of the mult-sequence algorthm (MSA). The algorthm proceeds n the order from (a) to (f). The clusters n magenta represent those already selected and those n cyan are to be examned n the next turn. more than two acheves better performance wth a more effcent algorthm of selectng NN canddates.. Proposed Method.. Overvew As mentoned n Sec., the key dea of the proposed method s to select NN canddates wthout comparng data or clusters. We apply the branch and bound algorthm to the problem to whch MSA s appled. Fgure, correspondng to Fg., provdes an overvew of the proposed method. In the fgure, paths from query (Q) to the centrods (C) n the orgnal space are drawn. The left and rght halves of the paths represent the squared dstances {d j } between the query and centrods n subspaces and, respectvely. d j shares ts ndexes wth C j. Then, an upper bound of the squared dstance s determned. The (f) 7

4 C C C C C C C C C C C C C C C C C C C C C C C C C C C C Max 7 Max Max d = d d = d = d d = d = d = d = = d d = d = = d d d = = = d d = Q C Q C Q = = d d C Q d = = = d d C = = d = d = d = d = d = d = (a) (b) (c) (d) Fgure : Proposed algorthm when the upper bound of the squared dstance s. The algorthm proceeds n the order from (a) to (d). Query Q d = d = d = d = Squared dstances between the query and centrods n subspace d = d = d = Centrods n the orgnal space C Squared dstances between the query and centrods n subspace Fgure : Overvew of the proposed method when the feature space s dvded nto two. It corresponds to Fg.. Paths from query (Q) to centrods (C) n the orgnal space are drawn. upper bound s ncremented throughout the process. All the paths whose total dstances are less than an upper bound are selected n the followng process. Fgure llustrates the proposed algorthm when the upper bound of the squared dstance s. The algorthm proceeds n the order from (a) to (d). In (a), when the upper bound s set at, the path wth d = s mmedately removed because the path dstance s larger than the upper bound. Hereafter, each path n the subspace (left half) s examned. In (b), the path wth d =s examned. As the dfference between the path dstance and the upper bound s 7, the paths wth d =and d =on the subspace (rght half) are selected as NN canddates. In (c), the path wth d =s examned. As the dfference between the path dstance and the upper bound s, the paths wth d = and d =on the subspace (rght half) are selected as NN canddates. In (d), the path wth d =s examned. As the dfference between the path dstance and the upper bound s, the paths wth d =and d =on the subspace (rght half) are selected as NN canddates. Fnally, the same results are obtaned as n Fg.... Preparaton To explan the proposed method n more detal, some defntons and preparatons are gven. Let us assume both query, denoted by q, and data are represented as vectors. Let N and D be the number of data, and the dmensonalty of query and data, respectvely. We apply the prncpal component analyss (PCA) to the data so as to mnmze the estmaton error of dstance, and obtan the largest u prncpal components (egenvalues) l,l,...,l u n descendng order. Let V = [ v v... v u ] be the matrx of egenvectors correspondng to the u egenvalues. u corresponds to the dmensonalty of the mult-dmensonal hash table. We ntroduce a way to automatcally determne u later. Hereafter a cluster refers to a bucket... Indexng and parameter tunng PQ dvdes the feature space nto p-dmensonal subspaces,.e., a u-dmensonal vector x n the subspace spanned by u egenvectors s represented as x = {x,...,x m }, where m = u/p. Then, k-means clusterng s appled to each p-dmensonal subspace. Wth the excepton of p, we present an automatc parameter tunng algorthm. Let us begn wth the automatc parameter selecton for k-means clusterng. The purpose here s to dvde data nto k clusters n the -th subspace, and determne {k } and the centrods of the clusters n subspaces. Let x s be the -th subvector of the s-th datum, and C j be the j-th centrod n the -th subspace. Then, the quantza-

5 Algorthm : Adaptve quantzaton wth automatc parameter tunng. Ths algorthm fnds the number N bkt of buckets whch are closest to the number N of data. Input: p, D, N Output: N bkt, {k }, {Cj} m D/p ; k, for =,...,m; N bkt ; Calculate the ntal {Cj}, whch are means n subspaces; repeat // Preserve the current values of N bkt, {k } and {Cj} N bkt N bkt; {k } {k }; {Cj } {Cj}; // Increment by the number of clusters n the subspace havng the largest error Calculate {E }; t argmax E ; k t k t +; Update the centrods n the t-th subspace; // The number of buckets s determned by the product of the number of clusters k ; untl N bkt >N; f N/N bkt s closer to than N/N bkt then // Revert to the prevous values of N bkt, {k } and {Cj} N bkt N bkt; {k } {k }; {Cj} {C j }; end 7 N bkt m ton error E n the -th subspace s defned as N ( E = x s Cq (x s)), () s= where q (x) s the assgnment functon gven as ( ) q (x) =argmn x C j. () j Because a large quantzaton error causes a large estmaton error n dstance, we take up the strategy of mnmzng the largest quantzaton error E max n the subspaces. Algorthm outlnes the proposed algorthm. It ncreases the number of clusters n the subspace havng the largest quantzaton error, and ends when the total number of buckets approaches the number of data. As a result, the feature space s dvded nto N bkt buckets, whch corresponds to the hash sze of the mult-dmensonal hash table. We emprcally found the stoppng condton from experments over dfferent numbers of artfcal data followng the normal and unform dstrbutons and real ones such as SIFT and GIST features. The algorthm also determnes the dmensonalty u of the mult-dmensonal hash table. If the number k of clusters n a subspace s, the quantzaton error n the subspace s less than E max. In such a case, the subspace need no dvdng. Assumng the egenvalues are arranged n descendng order, there exsts m whch satsfes k > for =,...,m and k = for = m +,..., D/p. Then, gnorng the latter half ( m +), u s set to m p. Algorthm : Incremental search regon algorthm to effcently obtan the mnmum sze c of NN canddates. Input: {d j}, c Output: a lst of buckets contanng NN canddates n ; L ; U m b= mnj db j +Δ; // Pre-calculate the mnmum and maxmum partal bucket dstances n from the ( +)-thtothem-th subspaces for =to m do d mn m b=+ mnj db j; d max m b=+ maxj db j; end whle n<cdo // Fnd new NN canddates 7 n n+ CrcutBucketsFuncton(, ); // Extend the search regon L U; U U +Δ; end Algorthm : Functon CrcutBucketsFuncton(, d). Input: : ndex of subspace, d: partal bucket dstance Output: n: the number of NN canddates f <mthen // Fnd an ndex n the -th subspace satsfyng the condton, and then contnue the search n the ( +)-th subspace forall the j do f L d + d j + d max & d + d j + d mn <Uthen h j; CrcutBucketsFuncton( +,d+ d j); end 7 end else // Return the bucket ndexed by the m ndexes forall the j do f L d + d m j & d + d m j <Uthen Return the bucket whose hash value s H = {h,h,,h m} and the number of data n the bucket as n; end end end.. Effcent selecton of NN canddates Let c be the number of NN canddates requred. We propose an algorthm to select at least c NN canddates wth the smallest estmated dstances n a step-by-step manner. Algorthms and show the detaled procedure, a part of whch s already presented n Sec... Algorthm fnds buckets whose dstances are n the range between the lower bound L and upper bound U of the estmated dstance, callng recursvely Algorthm. The ntal values of L and U are and the squared dstance between the query and the nearest bucket, respectvely. As long as the number n of the

6 NN canddates obtaned so far s less than c, the process s repeated wth updated search regons. Δ s the ncrement for L and U. We used / of the sum of the egenvalues as Δ.. Experments We performed experments to compare the proposed method () wth representatve ANNS methods, whch nclude [] and [] as hash-based methods and randomzed kd-tree () [] and herarchcal k- means () [] as tree-based methods from the components of FLANN []. All the methods were mplemented n the C++ language. We mplemented the hash-based methods by ourselves, referrng to the MATLAB verson of and C++ source code of., and share most of the source code. Ths avods dfferences n mplementaton and makes for a farer comparson. For the tree-based methods, we used the C++ source code of FLANN obtaned at the authors homepage. FLANN tself dd not work for large datasets. We expermentally explored the best parameters for each method, and determned ones whch acheved the best performance n the crteron, recall as a functon of computaton tme. The parameters used n the experments are summarzed n Table. We used the BIGANN database contanng bllon -dmensonal SIFT descrptors [] and the Mllon Tny Images 7 contanng approxmately mllon - dmensonal GIST descrptors []. For the former, mllon, mllon and mllon datasets were used; data were used as queres. For the latter, thousand, mllon and mllon datasets were used; the frst data n the frst categores ( data n total) were removed from the datasets and used as queres. For both databases, the smallest dataset, whch s a subset of larger datasets, s used for tranng (parameter tunng). We employed servers where CPUs (AMD Opteron 7,.GHz, cores) and GB memory were nstalled. All data were stored on memory. Each program was executed as a sngle thread on a sngle core... Experment : Recall vs computaton tme We compared the methods n the crteron, recall as a functon of computaton tme. Computaton tme s defned as the average computaton tme requred to obtan an answer from the tme the query s gven. Accuraces for small Although LSH [], spectral hashng [], transform codng [] were examned, ther performance was too poor. Therefore, we have omtted them marusm/ndex.php/flann/flann computaton tmes were relatvely low because of lmtatons n measurng computaton tmes. Fgure shows expermental results on the SIFT and GIST datasets. They show that the proposed method performed much better than the others. Fgure (c) shows that was two tmes faster than and. tmes faster than n % recall, and. tmes faster than and. tmes faster than IV- FADC n % recall. Comparng Fgs. (a) to (c), the advantage of the proposed method became clearer as dataset sze ncreased. Ths shows scalablty of the proposed algorthm. In contrast, comparng Fgs. (d) to (f), the advantage of the proposed method does not change. Ths mght be due to the hgh dmensonalty of the feature vectors. The fact that and dd not scale s caused by relatvely large computatonal overhead. Lettng G be the number of clusters n the orgnal space, needs tme G for the dstance calculaton n selectng NN canddates. needs tme G for dstance calculaton, and tme G log G n sortng the dstances n the subspaces... Experment : Recall vs number of canddates We also compared the methods n the commonly used crteron, recall as a functon of the number of canddates. Wth ths crteron, we show the expermental results of hash-based methods. We selected the best parameters subject to the crteron, recall as a functon of computaton tme. Fgure shows that was also better than and IV- FADC n the standard crteron.. Conclusons In the approxmate nearest neghbor search (ANNS) process, only the selecton of nearest neghbor canddates has any leeway for mprovement. In ths paper, we ponted out the mportance of evaluatng the computatonal costs of ANNS methods. Because takng the computaton tme nto account s nherent n the ANNS problem, researches gnorng computaton tme have only a lmted mpact. We also ponted out that the state-of-the-art ANNS method,, does not capture the essence of NN canddate selecton. That s, although the goal of NN canddate selecton s smply fndng NN canddates, t employs unnecessary processes such as comparatve sort and heap. Fnally, we proposed a new ANNS method that takes nto account the cost of NN canddate selecton. The proposed method s based on the branch and bound algorthm whch produces a sgnfcantly more effcent search. We also presented an automatc parameter tunng algorthm whch, except for one parameter, can automatcally determne the best parameters. In an experment usng mllon SIFT features, the proposed method succeeded n reducng the computaton tme by one thrd compared wth the state-of-theart; the proposed method was two tmes faster n % recall, and. tmes faster n % recall.

7 Table : Parameters used n the experments. Methods Parameters SIFT M SIFT M SIFT M GIST K GIST M GIST M log C, P,,,,,, log C log C No. of trees k Not executable 7.. (a) SIFT M 7. (b) SIFT M 7. (c) SIFT M 7.. (d) GIST K 7. (e) GIST M 7. (f) GIST M Fgure : Experment : Recall vs Computaton tme. Acknowledgment Ths work was supported by JST CREST project, JST Funds for ntegrated promoton of socal system reform and research and development, JSPS KAKENHI #, and JST A-STEP #ASZH. References [] A. Andon and P. Indyk. Near-optmal hashng algorthms for approxmate nearest neghbor n hgh dmensons. In Proc. FOCS,. [] A. Babenko and V. Lemptsky. The nverted mult-ndex. In Proc. CVPR,.,,, [] J. Brandt. Transform codng for fast approxmate nearest neghbor search n hgh dmensons. In Proc. CVPR,., [] M. Datar, N. Immorlca, P. Indyk, and V. S. Mrrokn. Localty-senstve hashng scheme based on p-stable dstrbutons. In Proc. th annual symposum on Computatonal geometry,., [] A. Gersho and R. Gray. Vector Quantzaton and Sgnal Compresson. Kluwer Acadmc Publshers,. [] Y. Gong and S. Lazebnk. Iteratve quantzaton: A procrustean approach to learnng bnary codes. In Proc. CVPR,. [7] Y. Gong, S. Lazebnk, A. Gordo, and F. Perronnn. Iteratve quantzaton: A procrustean approach to learnng bnary codes for large-scale mage retreval. IEEE Trans. PAMI,. [] P. Indyk and R. Motwan. Approxmate nearest neghbor: towards removng the curse of dmensonalty. In Proc. th Symposum on Theory of Computng,. [] M. Iwamura, T. Kobayash, and K. Kse. Recognton of multple characters n a scene mage usng arrangement of local features. In Proc. ICDAR,. [] M. Iwamura, T. Tsuj, and K. Kse. Memory-based recognton of camera-captured characters. In Proc. DAS,.

8 7 (a) SIFT M 7 e+ (b) SIFT M 7 e+ (c) SIFT M 7 (d) GIST K 7 e+ (e) GIST M 7 e+ (f) GIST M Fgure : Experment : Recall vs Number of canddates. [] H. Jégou, M. Douze, and C. Schmd. Packng bag-offeatures. In Proc. ICCV,. [] H. Jégou, M. Douze, and C. Schmd. Product quantzaton for nearest neghbor search. IEEE Trans. PAMI, ():7,.,, [] Y. Ke, R. Sukthankar, and L. Huston. Effcent near-duplcate detecton and sub-mage retreval. In Proc. ACM Multmeda,. [] K. Kse, K. Noguch, and M. Iwamura. Robust and effcent recognton of low-qualty mages by cascaded recognzers wth massve local features. In Proc. WS-LAVD,. [] W. Kong and W.-J. L. Isotropc hashng. In Proc. NIPS,. [] B. Kuls and T. Darrell. Learnng to hash wth bnary reconstructve embeddngs. In Proc. NIPS,. [7] R.-S. Ln, D. A. Ross, and J. Yagnk. SPEC hashng: Smlarty preservng algorthm for entropy-based codng. In Proc. CVPR,. [] D. G. Lowe. Dstnctve mage features from scale-nvarant keyponts. IJCV, (),. [] Q. Lv, W. Josephson, Z. Wang, M. Charkar, and K. L. Mult-probe LSH: effcent ndexng for hgh-dmensonal smlarty search. In Proc. VLDB, 7. [] M. Muja and D. G. Lowe. Fast approxmate nearest neghbors wth automatc algorthm confguraton. In Proc. VISS- APP. INSTICC Press,.,, [] D. Nstér and H. Stewénus. Scalable recognton wth a vocabulary tree. In Proc. CVPR,.,, [] M. Norouz and D. J. Fleet. Mnmal loss hashng for compact bnary codes. In Proc. ICML,. [] A. Olva and A. Torralba. Modelng the shape of the scene: A holstc representaton of the spatal envelope. IJCV,,. [] C. Slpa-Anan and R. Hartley. Optmsed kd-trees for fast mage descrptor matchng. In Proc. CVPR,., [] K. Takeda, K. Kse, and M. Iwamura. Memory reducton for real-tme document mage retreval wth a mllon pages database. In Proc. CBDAR,. [] A. Torralba, R. Fergus, and W. T. Freeman. mllon tny mages: a large dataset for non-parametrc object and scene recognton. IEEE Trans. PAMI, (): 7,. [7] E. Tuncel, H. Ferhatosmanoglu, and K. Rose. VQ-ndex: an ndex structure for smlarty searchng n multmeda databases. In Proc. ACM Multmeda,. [] Y. Wess, R. Fergus, and A. Torralba. Multdmensonal spectral hashng. In Proc. ECCV, Part V (LNCS 77),. [] Y. Wess, A. Torralba, and R. Fergus. Spectral hashng. In Proc. NIPS,., [] H. Xu, J. Wang, Z. L, G. Zeng, S. L, and N. Yu. Complementary hashng for approxmate nearest neghbor search. In Proc. ICCV,. [] L. Zhang, Y. Zhang, J. Tang, K. Lu, and Q. Tan. Bnary code rankng wth weghted hammng dstance. In Proc. CVPR,.