Real-Time Feature Descriptor Matching via a Multi-Resolution Exhaustive Search Method

Transcription

1 297 Rea-Time Feature escriptor Matching via a Muti-Resoution Ehaustive Search Method Chi-Yi Tsai, An-Hung Tsao, and Chuan-Wei Wang epartment of Eectrica Engineering, Tamang University, New Taipei City, Taiwan Emai: chiyi_tsai@mai.tu.edu.tw; @s0.tu.edu.tw; @s00.tu.edu.tw Abstract Feature descriptor matching pays an important roe in many computer vision appications. This paper presents a nove fast inear ehaustive search agorithm combined with a muti-resoution candidate eimination technique to dea with this probem efficienty. The proposed agorithm is inspired from the eisting mutiresoution image retrieva approaches, but reeasing the requirement on a norm-sorted database with pre-computed muti-resoution tabes. This heps to increase the appicabiity of the proposed method. Moreover, the computations of candidate eimination are fuy performed using a simpe L distance metric, which is abe to speedup the entire search process without oss of accuracy. This property eads to an accurate feature descriptor matching agorithm with rea-time performance, which wi be vaidated in the eperiments by testing with the matching of SURF descriptors. Inde Terms descriptor matching, inear ehaustive search, L norm pyramid, muti-resoution eamination I. INTROUCTION Feature descriptor matching is one of the critica stages in various computer vision appications, especiay in image retrieva, image stitching, 3 pose estimation, and muti-view reconstruction, etc. As a resut, the study of image feature detection, description and matching has become an active research topic in the ast years. This paper focuses on the ast issue, which typicay can be treated as a nearest neighbor search probem in mutipe dimensions [] and thus can be resoved by using a conventiona nearest neighbor search technique [2], [3]. eveoping an efficient and accurate feature descriptor matching agorithm is important as it greaty affects the efficiency and accuracy of the foowing vision agorithms. This issue hence motivates us to design a new feature descriptor matching agorithm achieving fast search speed with high matching accuracy. Current feature descriptor matching methods can be roughy divided into distance metric and space partitioning approaches. The former empoys a distance metric to measure the simiarity between a query descriptor and a sampe descriptor in a given data set, and the ater empoy a tree data structure to efficienty speedup the search of nearest neighbors. The simpest Manuscript received October 25, 202; revised March 0, 203. distance metric method is we nown as the inear ehaustive search (LES) method, so-caed the bruteforce method, which usuay empoys Eucidean distance metric to search for a nearest neighbor. The LES method often provides the best matching resut, but it becomes inefficient as the size of database or the dimension of feature descriptor increasing dramaticay. By contrast, the space partitioning approaches are efficient for searching mutipe nearest neighbors in a arge database. However, it is we nown that the performance of space partitioning methods is degraded eponentiay as the dimension of feature descriptor growing up, caed the curse-of-dimensionaity [4]. In practice, the feature descriptor is usuay of high dimensionaity, i.e., the dimension of the popuar SIFT [5] and SURF [6] descriptors are 28 and 64, respectivey. This probem highights the importance to deveop an efficient and accurate descriptor matching agorithm for feature matching in high dimensiona spaces. To achieve this purpose, this paper presents a fast muti-resoution ehaustive search agorithm to efficienty search the best match in a high dimensiona space. The proposed agorithm is inspired from the eisting muti-resoution image retrieva approaches [7], [8], but reeasing the requirement on a norm-sorted database with precomputed muti-resoution tabes. This heps to increase the appicabiity of the proposed method. Moreover, the proposed muti-resoution candidate eimination method is abe to detect and remove a non-candidates from a arge candidate ist via a simpe L distance computation, aowing speeding up the entire search process without oss of accuracy. This property eads to an efficient and accurate feature descriptor matching agorithm achieving rea-time performance in practica appications, which wi be vaidated in the eperiments by testing with the matching of SURF descriptors. escriptor Number Nq uery Set escriptor escriptor 2 escriptor 3 escriptor Nq-2 escriptor Nq- escriptor Nq imension Matching Set ata Set escriptor escriptor 2 escriptor 3 escriptor Nd-2 escriptor Nd- escriptor Nd imension escriptor Number Figure. The feature descriptor matching probem considered in this study. That is, given a query and a data set, the goa is to find a matching set that records the descriptor-matching pairs in both given sets. Nd doi:0.4304/jsw

2 298 II. PROBLEM STATEMENT Fig. shows the feature descriptor matching probem considered in this study, which is aso caed the nearest neighbor search probem. Given a query set X, X,, } and a data set Y, Y,, }, { 2 X N q { 2 Y N d where X i, Yj, i ~ N and q j ~ N, denote the - d dimensiona feature descriptors in query and data sets, respectivey. The goa is to find a matching set M {(, y),( 2, y2),,( m, ym)}, where n, y n with n ~ m are nonzero positive vaues indeing the matching points in both given sets. For instance, Fig. shows three pairs of matching points in the query and data sets, and the matching set is thus given by M {(, 3),(3, N d ),( Nq2,)}. To find the matching set, one of the most widey used methods is the LES method (so-caed the brute-force method) that searches for a matching point by evauating the simiarity between a query descriptor and a unmatched descriptors in the data set. The simiarity between two descriptors is usuay measured by Eucidean distance (or L 2 distance) defined as d ( X, X Y y () i i where X and Y are two descriptors in the query and data sets, respectivey. Fig. 2 shows the pseudo-code of the traditiona LES agorithm. Note that as suggested from the previous wors [9], we aso empoy a ratio test on descriptor distances at the ast step of matching process to reject ambiguous matches. Athough the LES agorithm provides a simpe way to find the matching set, the runtime of this method is of order O(N q N d ), eading to an inefficient agorithm as the dimension and ength N q or N d growing dramaticay. The Traditiona Linear Ehaustive Search Agorithm Input: A query set Ω ={X,X 2,,X Nq } and a data set Ω ={Y,Y 2,,Y Nd } Output: A match set Ω M. Initiaize: Cear the match set Ω M ={0}; Set a matching counter c=0; Set a distance ratio α=0.65; 2. for i=:n q do 3. d =d 2 =FLT_MAX; c =i; 4. for j=:n d do 5. dist=d 2 (X i,y j ); 6. if dist<d then 7. d 2 =d ; d=dist; y c =j; 8. ese if dist<d2 then 9. d 2 =dist; 0. end if. end for 2. if d <αd 2 then 3. Push ( c,y c ) into match set Ω M ; c=c+; 4. end if 5. end for 6. Return: match set Ω M FLT_MAX: Maimum vaue of a foating-point variabe Figure 2. Pseudo-code for the traditiona inear ehaustive search agorithm. Unie the traditiona LES agorithm, another commony used search method is the space partitioning method that empoys tree-ie structures, such as -d trees, to iterativey reduce the search space, efficienty i increasing the nearest-neighbor search performance in a arge data set. However, the author in [0] pointed out that the LES method is faster than the space partitioning one when og N. Since such high dimensionaity d (i.e., 25) usuay occurs in practica appications, this issue highights the importance of deveoping a fast ehaustive search method to dea with descriptor matching in high dimensiona spaces. This study thus proposes a nove fast muti-resoution ehaustive search method to achieve this purpose efficienty and effectivey. III. THE PROPOSE FEATURE MATCHING ALGORITHM This section presents the proposed fast ehaustive search agorithm, which is deveoped by combining the LES agorithm with a L -norm based muti-resoution candidate eimination technique. A. Muti-Resoution Candidate Eimination In this subsection, a nove muti-resoution eimination method inspired from the inequaity property of L norm pyramid [7], [8] is presented to efficienty speedup the matching process in high dimensiona spaces. We first define the L distance between two descriptors such that d ( X, X Y i y (2) i where X and Y. According to [7], the computation of L distance can be etended to a mutiresoution case, aowing improving the ehaustive search performance in a muti-eve manner. Suppose that the dimension of the feature descriptor satisfies = 2 L for any nonzero positive vaue L. A L norm pyramid associated with a feature descriptor X can be defined as a sequence of muti-resoution descriptors ( 0) () ( L) ( L) { X X X X } with X (L) = X, where T X [ 2 ] for 0 L denotes a hafresoution version of X (+) formed by successivey 2 summing two neighboring eements of X (+) such that for i 2 (Fig. 3). Then, an important i 2i 2i inequaity reated to the L distance between two descriptors can be derived based on the triange inequaity such that ( L) () d ( X, d ( X, d ( X, d ( X, ) (3) where i Y d ( X, X Y is the L distance between the ow-resoution version of X and Y. The inequaity property in Eq. (3) provides a significant cue to speedup the ehaustive search operation. That is, given a query descriptor X and a minimum distance of matching criteria d min, it is possibe to eiminate a matching candidates in Ω whose L distance in ow-resoution domain satisfies ( ) d ( X, d. Since the computationa cost of L min distance at ower eves is much ess than that of L 2 distance in fu-resoution domain, the run-time of the ehaustive search can be improved dramaticay via this muti-resoution eimination method. We thus combine this muti-resoution eimination method with the LES

3 299 method to form an efficient ehaustive search agorithm with ow computationa compeity. X (L-) X (L-) X (2) 2 (L-) X () X () 2 () (2) (2) (2) 2 (2) 3 4 (L-) (L-) /2- / Leve 0 Leve Leve 2 Leve L- Leve L Figure 3. The L norm pyramid of a feature descriptor X with dimension = 2 L. B. Minimum istance etermination As mentioned in the previous subsection, the mutiresoution eimination method requires a minimum distance d min as the matching criteria to eiminate any matching candidate in Ω for a given feature descriptor X in Ω. To determine the vaue of d min, the authors in [7] first seect an initia best match Y ibm whose norm Y is cose to X. Net, the vaue of d min is initiay computed as d min = d (X, Y ibm ) and is updated when a better match is found in the rest of candidates. This method is simpe, but ony effective in a norm-sorted database constructed in an off-ine preprocess. In many rea-time appications, however, the data set Ω is obtained on-ine and is not norm-sorted. This probem therefore motivates us to deveop a new minimum distance computation method to compute an appropriate vaue of d min for the ehaustive search in a non-sorted data set. Suppose that a muti-resoution descriptors of both query and data sets are buit and stored in 2L mutiresoution tabes. Let T and ibm T, 0 L, denote the muti-resoution tabes for the query and data sets, respectivey. Unie the previous method in [7], the proposed method aims to determine an appropriate minimum distance based ony on the owest resoution tabes T and 0 T, significanty reducing the 0 computationa burden of the minimum distance determination. Moreover, as the initia vaue of d min determined, its vaue wi eep as a constant for a query descriptors during the candidate eimination process, eading to faster ehaustive search performance but oss of matching accuracy. To maintain the matching accuracy, we remain the ratio test on two L 2 distances at the ast step as suggested in the LES method. This is another main difference between the proposed method and the previous wors. The concept of the proposed minimum distance computation method is simpe. First, we tae a subset of Ω as a training set. The vaue of d min is then computed as the average of the L distance at Leve 0 between each descriptor in the seected subset and a descriptors in Ω. More specificay, the vaue of d min is obtained by N N d d dmin ( X i, Yj ) with N N q (4) NNd i j where means the argest integer ess than or equa to, and β denote a nonzero positive number satisfying 0 β to define the subset of Ω. Empiricay, we found that setting the vaue of β as 0.25 may obtains the best candidate eimination performance with ess computationa burden. We thus choose β = 0.25 as the defaut setting for the proposed minimum distance computation method. The Proposed Fast Muti-resoution Ehaustive Search Agorithm Input: A query set Ω ={X,X 2,,X Nq } and a data set Ω ={Y,Y 2,,Y Nd } Output: A match set Ω M. Initiaize: Cear the match set Ω M ={0}; Set a matching counter c=0; Set a distance ratio α=0.65; 2. Buid muti-resoution tabes T 0 ~T L- for the query set ; 3. Buid muti-resoution tabes T 0 ~T L- for the data set ; 4. etermine the minimum distance d min based ony on the tabes T 0 and T 0 ; 5. for i=:n q do 6. d =d 2 =FLT_MAX; c =i; 7. for j=:n d do 8. if d () (X i,y j )>d min, =0~L-2, then 9. continue; 0. ese if d (L-) (X i,y j )>d min then. dist=d 2 (X i,y j ); 2. if dist<d2 then 3. d 2 =dist; 4. end if 5. continue; 6. ese 7. dist=d 2 (X i,y j ); 8. if dist<d then 9. if d<d2 then 20. d 2 =d ; 2. end if 22. d=dist; y c =j; 23. ese if dist<d2 then 24. d 2 =dist; 25. end if 26. end if 27. end for 28. if d <αd 2 then 29. Push ( c,y c ) into match set Ω M ; c=c+; 30. end if 3. end for 32. Return: match set Ω M Figure 4. Pseudo-code for the proposed muti-resoution ehaustive search agorithm. C. The Proposed Agorithm Fig. 4 shows the pseudo-code of the proposed fast muti-resoution ehaustive search agorithm, which combines the LES method with the muti-resoution candidate eimination method. After initiaization, the muti-resoution tabes T and T with 0 L are buit for a descriptors in both query and data sets. The vaue of d min is then computed by using Eq. (4). The foowing ehaustive search procedure is simiar to the LES method, ecept that we appy the muti-resoution

4 2200 eamination process at the ower eves before the L 2 distance eamination at the highest eve. This heps to achieve high search speed without oss of accuracy, even in high dimensiona spaces. The search performance of the proposed method wi be vaidated in the eperiments. TABLE I. PERFORMANCE COMPARISONS OF THE PROPOSE METHO WITH TWO EXISTING METHOS Image- Pair No. Inier ratio (%) Search Method LES -d tree Proposed method Search Inier Search Inier Search time ratio time ratio time (ms) (%) (ms) (%) (ms) Avg IV. EXPERIMENTAL RESULTS In the eperiments, we combined the SURF eypoint detector [6] with the LES, -d tree, and the proposed methods to evauate their feature matching performance in a high dimensiona space ( = 64). Two performance metrics were used to evauate the matching performance of each competing method, incuding an inier ratio and a search time metrics. The former measures the ratio of the tota inier-point number to the tota matching-point number, indicating the matching accuracy of the search agorithm. By contrast, the ater measures the overa search time (in miisecond) spent on the matching process, indicating the computationa efficiency of the search agorithm. An efficient feature matching agorithm shoud behave a higher inier ratio with a sma search time. Note that we appied an eisting outier remova method [] to etract a iniers and compute the inier ratio for each competing method. Five test image-pairs were used in out testing, and Tabe I records the eperimenta resuts. It is cear from Tabe I that the proposed method provides the best inier ratio and the smaest search time, foowed by the LES and -d tree methods. Moreover, the inier ratio of the proposed method is about 95%, and its search performance is about nineteen times the LES method and about sity times the -d tree method. Thus, the proposed method is more suitabe to meet the requirement of rea time appications. Fig. 5 shows the feature matching resuts of the test image-pair No. 5. In Fig. 5, the green and red ines indicate, respectivey, the inier and outier matches after the outier remova process. We can see from Fig. 5 that the LES and -d tree methods produce more outiers in their matching resuts when compared to the proposed method. The above eperimenta resuts therefore vaidate the search accuracy and computationa efficiency of the proposed feature matching agorithm. (a) (b) (c) Figure 5. SURF descriptor matching resuts of test image-pair No. 5 obtained by (a) the LES method, (b) the -d tree nearest neighbor search method, and (c) the proposed method. In the above images, the green and red ines indicate, respectivey, the inier and outier matches after an outier remova process [].

5 220 V. CONCLUSIONS AN FUTURE WORK This paper proposes an efficient and accurate mutiresoution ehaustive search agorithm that achieves high feature descriptor matching accuracy with rea-time performance. The proposed agorithm consists of a L - norm based muti-resoution candidate eimination technique, aowing efficienty removing a noncandidates in a arge candidate ist with ess computationa burden. A nove minimum distance determination method is aso proposed to decide a minimum threshod vaue for eiminating candidates. Eperimenta resuts vaidate the performance of the proposed method by comparing with two eisting search methods, quantitativey and effectivey. In the future, combination with a SIFT detector wi be further investigated. ACKNOWLEGMENT This wor was supported by the Nationa Science Counci of Taiwan, R.O.C. under grant NSC E REFERENCES [] S. A. Nene and S. K. Nayar, A simpe agorithm for nearest neighbor search in high dimensions, IEEE Transactions on Pattern Anaysis and Machine Inteigence, vo. 9, pp , September 997. [2] J. L. Bentey, Mutidimensiona binary search trees in database appications, IEEE Transactions on Software Engineering, vo. 5, pp , Juy 979. [3] T. Uemiya, Y. Matsumoto,. Koizumi, M. Shishibori, and K. Kita, Fast mutidimensiona nearest neighbor search agorithm based on eipsoid distance, Internationa Journa of Advanced Inteigence, vo., pp , November [4] R. Weber, H. Sche, and S. Bott, A quantitative anaysis and performance study for simiarity search methods in high-dimensiona spaces, in Proc. 24 th Very Large ata Bases, New Yor, USA, 998, pp [5]. G. Lowe, istinctive image features from scaeinvariant eypoints, Internationa Journa of Computer Vision, vo. 60, pp. 9-0, November [6] H. Bay, A. Ess, T. Tuyteaars, and L. V. Goo, Speededup robust features (SURF), Computer Vision and Image Understanding, vo. 0, pp , June [7] S. C. Song, M. J. Kim, and J. B. Ra, A fast mutiresoution feature matching agorithm for ehaustive search in arge image databases, IEEE Transactions on Circuits and Systems for Video Technoogy, vo., pp , May 200. [8] S. C. Song, and J. B. Ra, Mutiresoution descriptor matching agorithm for fast ehaustive search in normsorted databases, Journa of Eectronic Imaging, vo. 4, pp. -7, October-ecember [9] P. Morees and P. Perona, Evauation of features detectors and descriptors based on 3 objects, Internationa Journa of Computer Vision, vo. 73, pp , Juy [0] J. M. Keinberg, Two agorithms for nearest-neighbor search in high dimensions, in Proc. 29 th ACM Symposium on Theory of Computing, Teas, USA, 997, pp [] C. Osson, A. Erisson, and R. Hartey, Outier remova using uaity, in Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 200, pp Chi-Yi Tsai was born in Kaohsiung, Taiwan, Repubic of China in 978. He received the B.S. and M.S. degree in eectrica engineering from Nationa Yunin University of Science and Technoogy, Yunin, Taiwan, in 2000 and 2002, respectivey. He received the Ph.. degree in eectrica and contro engineering, at the Nationa Chiao Tung University, Taiwan, in He is currenty an assistant professor in department of eectrica engineering, Tamang University, Taiwan. His research interests incude image processing, coor image enhancement processing, visua tracing contro for mobie robots, visua servoing and computer vision. modeing. An-Hung Tsao was born in Taipei, Taiwan, Repubic of China in 989. He received the B.S. degree in eectrica engineering from Tamang University, Taiwan, in 20. He is currenty pursuing the M.S. degree in eectrica engineering at the Tamang University, Taiwan. His research interests incude image processing and computer vision. Chuan-Wei Wang was born in Taipei, Taiwan, Repubic of China in 988. He received the B.S. degree in eectrica engineering from Tamang University, Taiwan, in 20. He is currenty pursuing the M.S. degree in eectrica engineering at the Tamang University, Taiwan. His research interests incude point coud registration and 3