A Novel Image Matching Method Based on SIFT

Transcription

1 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp Sensors & Transducers 04 by IFSA Publshng, S. L. A Novel Image Matchng Method Based on SIFT Yuan-Sheng LIN, * Gang XU, Mng JIANG, Peng ZHOU, Juan-Juan JIANG Anhu Provncal Lab of Detecton Technology and Energy Savng Devces, Anhu Polytechnc Unverst Wuhu 40, Chna * E-mal:xgwml@63.com Receved: May 04 /Accepted: 30 May 04 /Publshed: 3 May 04 Abstract: The research on mage matchng method has been one of the man research focuses n recent years. In order to reduce the number of ncorrect matchng ponts and mprove the performance of mage matchng n the mage processng system, ths paper presents a novel mage matchng method based on Scale Invarant Feature Transform (SIFT) algorthm and cosne smlarty. The method extracts the SIFT keyponts of mage and mage frstl and then makes the mage matchng by computng the cosne smlarty of the extracted SIFT feature vectors. Extensve expermental results on the Mddlebury Stereo Datasets show that the novel mage matchng method reduces the error matchng and mproves the matchng effectvel wth strong robustness and a better matchng effect. Copyrght 04 IFSA Publshng, S. L. Keywords: Image matchng, SIFT, Cosne smlart Keyponts, Feature.. Introducton Wth the constant development of dgtal mage processng and machne vson technologes, the mage matchng research s gradually becomng one of the hot topcs for researchers. Image matchng refers to usng effectve methods to analyze the consstences or lkelhoods n two or more mages of the same object whch are acqured n the dfferent vson condtons, and fnd the most homologous mage ponts among these mages []. Davd G. Lowe put forward SIFT algorthm perfectly n 004 after 5-year perfecton and summar and t s appled to extract local feature wdely; the SIFT features are nvarant to mage rotatons, llumnaton changes, scale changes and so on [-3]. The essence of mage matchng s to establsh the correspondng relatonshps between the features whch are extracted from the two or more two-dmensonal mages got from dfferent angles for 76 the same object. So mage matchng methods based on SIFT algorthm become possble and have recently obtaned effectve results, and some mproved SIFT mage matchng algorthms even meet the real-tme requrements [4-0]. It s rare that the SIFT method wth the matchng strategy based on K-means s used systematcally n face recognton n [4], and ther expermental results demonstrate the robustness of SIFT features to expresson, pose and accessory varatons. Image matchng method based on Harrs-SIFT, an mproved SIFT algorthm n stereo vson s researched n [5] and t s appled n the experment on bnocular stereo vson, expermental results demonstrate the effectveness of the method. An mprovng SURF mage matchng approach by usng supervsed learnng and reducng some SURF features was presented n [6], the expermental results show that the proposed method reduces the matchng tme successfully. Alexandra Glnsky and Lh Zelnkhttp://

2 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp Manor present the SIFTpack: a compact way of storng SIFT descrptors, and they have shown that the SIFTpack representaton saves not only storage space, but runtme for matchng between SIFTs n [7]. Meanwhle, some lteratures propose mage matchng methods based on mproved SIFT algorthm and have some experment or applcaton effects, whch get matched keyponts based on Eucldean dstance of the SIFT feature vectors [3, 8-0]. Image matchng algorthms based on SIFT and Eucldean dstance have been proven successfully n recent years. However, the mproved algorthms above mentoned have some dsadvantages, such as the feature utlzaton ratos are not very hgh, wth further rooms for mprovement n the matchng tme and the problems of repeated or error matchng. In ths paper, we presented a novel mage matchng method based on SIFT by usng cosne smlarty [] and the expermental results carred out on the Mddlebury Stereo Datasets demonstrate the usefulness of the method n mage matchng.. Key-feature-pont Extracton Based on SIFT Before the process of mage matchng, we should extract key-feature-ponts by usng the SIFT algorthm. The detal SIFT mage feature extracton process s shown n the followng Fg.. In order to detect stable key-feature-ponts more effectvel Lowe has proposed usng Dfference of Gaussan (DOG) functon shown n Equaton (3) to detect the scale-space extrema. D( σ ) ( G( kσ) G( σ)) I (, (3) L( kσ ) L( σ ) The SIFT key-feature-ponts are made up of the local extrema of D( σ); n order to detect the maxma and mnma ponts successfull each sample pxel s compared to ts 6 adjacent pxels n the current and neghborng scales.. Key-feature-pont Localzaton As the DOG operator s so senstve to edge responses and nose that once the entre key-featurepont canddates have been found, curve fttng s needed to mprove the stablty of the key-featureponts. The SIFT algorthm uses three-dmensonal quadratc functon shown n Equaton (4) to locate the key-feature-ponts: T D T D D( X ) D + X + X X, (4) X X where equaton (4) s the Taylor expanson of the scale-space functon, D( σ), and take the dervatve of D(X) respect to X and set t to zero, we can get: Fg.. Key-feature-pont extracton based on SIFT process. X - D D -, (5) X X.. Scale-space Constructon and Extrema Detecton The scale-space of an mage, L( σ), s defned as the convoluton of a varable-scale Gaussan, G( σ), wth the orgnal mage, I(. Therefore, the functon, L( σ), s defned as follows [3]: L( σ ) G( σ) I(, () where I( s a functon of the D orgnal mage, ( represents the pxels poston; * s the convoluton operaton; G( σ) s a Gaussan template functon; σ s the standard devaton of the Gaussan dstrbuton, also known as the scale-space factor, ts value drectly affects the degree of the mage blur; and ( x + y ) / σ G( σ ) e, () πσ by substtutng Equaton (5) nto (4), and we can get: T D D( X ) D + X, (6) X The extrema wth D ( X ) < were rejected n the experments. As DOG functon wth edge response strongl the rato γ of the prncpal curvatures (the largest and the smaller) along the edge s relatvely large. For our experments (γ0), all the key-feature-pont canddates layng on edges wth a prncpal curvatures rato value more than γ must be dscarded. In the experments, the prncpal curvatures were computed from a Hessan matr H: D H D xx xy D D xy yy, (7) 77

3 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp Snce the egenvalues, α (the largest one) and β (the smaller one) of H are proportonal to the prncpal curvatures of D, we can get γ α/β. Then: Tr( H ) ( Dxx + Dy Det( H ) D D ( D ) xx yy ( α + β ) αβ xy ( γ + ) γ, (8) As we know, (γ+) /γ ncreases wth γ when γ. Therefore, we only need to check: Tr( H ) ( γ + ) < Det( H ) γ.3. Orentaton Assgnment, (9) Only assgn an orentaton to each key-featurepont, the mage matchng method based on SIFT algorthm can be nvarant to mage rotaton. The gradent magntude, m(, and orentaton, θ(, of each mage, L( are calculated from the Equaton (0): m( ( L( x +, L( x, ) + ( L( y + ) L( y )) L( y + ) L( y ) θ ( arctan L( x +, L( x,, (0) And now, the SIFT feature ponts wth locaton, scale and orentaton are generated..4. Key-feature-pont Descrptor After extractng the SIFT feature ponts wth locaton, scale and orentaton, etc. The followng step s to generate descrptors for the key-featureponts. The descrptor used n our experments s a 8-element normalzed feature vector for each keyfeature-pont, whch s nvarant to some external changes such as llumnaton changes etc. 3. The Novel Image Matchng Method The SIFT algorthm s wdely used n mage matchng methods for that the SIFT feature s nvarant to mage rotatons, llumnaton or scale changes [-3]. Eucldean dstance, the most common dstance measurement standard, has been used as a matchng rule n many mproved mage or stereo matchng algorthms based on SIFT, and some mproved results have been acheved n the experments [3, 8-0]. However, some mprovements of the above mproved algorthms are not partcularly outstandng; there are further rooms for mprovement n the matchng tme and some msmatches or repeated-matches. In ths paper, we present a novel SIFT mage matchng method based on SIFT and cosne smlarty. Cosne smlart namely cosne dstance, s a measure of smlarty between two space vectors by measurng the cosne of the ncluded angle between them. The cosne smlarty between the vectors x and y s defned as []: x y cos θ, () x y where the vectors x and y must have the same dmenson,.e., x(x, x,, x n ) T and y(y, y,, y n ) T, by substtutng them nto (), gvng: cosθ, () n n x x y n Cosne of the two key-feature-pont vectors s larger,.e., the ncluded angle between them s smaller, the greater the smlarty between them, the greater the lkelhood of matchng. The descrptor used s a 8-element normalzed feature vector, therefore, n8 n our experments. Let mage be the reference mage, mage be the mage to be matched; and k be the number of keyfeature-ponts extracted from mage, k be the number of key-feature-ponts extracted from mage. And let A be a k 8 matr B be a k 8 matrx; then A, B by the followng equatons: a, a,8 A ak, a k,8 y b, b,8 B, bk, b k,8 (3) The mproved mage matchng strategy s as follows: Step.: Take the frst key-feature-pont vector of mage,.e., the frst row vector of A, [a,,,a,8 ], and get the cosne dstances between t and all the 78

4 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp observatons n the matrx B. Then take the nverse cosnes of the cosne dstances, and pck the smallest ncluded angle,, and the second smallest ncluded angle,. Step.: If / < d, then accept the frst orgnal match. Otherwse, exclude the orgnal match. Where, d s the determnaton threshold value and we assume d value n the range [0, ] n our experments. Step.3: The rest can be done n the same manner untl get the cosne dstances between all the rest of the row vectors of matrx A and the observatons n the matrx B, and check that the rato of the smallest ncluded angle and second smallest ncluded angle s below the threshold, d, n turn. In a word, we must ensure the k key-feature-ponts of amge and the k key-feature-ponts of mage be matched and get the number of the correct orgnal matches, j. Step.: From Step. to Step.3, we can obtan the j correct orgnal matched key-feature-ponts, and the j row vectors of B, whch are correct matched n the orgnal matchng compose the matrx B': ' ' b, b,8 B, ' ' b j, b j,8 (4) Step.: Smlarly as from Step. to Step.3, get the cosne dstances between each par of observatons n the matrx B' and A, and take the nverse cosnes of the matrx multplcaton results, then pck the smallest ncluded angle, 3, and the second smallest ncluded angle, 4, n turn. Step.3: If: Δ 3 < d, (5) 3/ 4 < d Then accept the fnal match/matches; otherwse, exclude the fnal match/matches. And we can get the number of the correct fnal matches, k. Where, d and d are the determnaton threshold values; and we assume that d values n the range [0, ], d (e.g ) values very close to 0 n our experments. All the prevous operatons have ntroduced the matchng strategy based on SIFT and cosne smlart and t can reduce msmatched or repeated matched key-feature-ponts, mprove the matchng rate effectvely. 4 Expermental Results and Analyss In order to valdate and analyze the feasblty of the mproved mage matchng strategy proposed n ths paper, we carred out some smulaton experments presented n ths secton n accordance wth the research scheme. In ths paper, all the experments are smulated on a personal computer, the PC bascs: Intel (R) Core (TM) S CPU@.80 GHz, the nstalled memory (RAM): 8.00 GB, the system type: 64-bt Wndows 7 Ultmate operatng system. And all of the experment programs are programmed by MATLAB R0a. All the mages used n our experments are from the Mddlebury Stereo Datasets, whch webste s They are Cones (sze 4 375) and Teddy (sze 4 375) [], Rendeer (sze ) and Books (sze ) [3, 4]. The smulaton experments are begun wth the key-feature-ponts generaton experments [5]; the smulaton results of experments on Cones, Tedd Rendeer and Books are shown n Fg., Fg. 3, Fg. 4 and Fg Fg.. Experments on Cones Table. Expermental data on Cones. SIFT + ED (94,353) Our Method (94,353)

5 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp Fg. 3. Experments on Teddy. Table. Expermental data on Teddy SIFT + ED (749,767) Our Method (749,767) Fg. 4. Experments on Rendeer. Table 3. Expermental data on Rendeer. SIFT + ED (630,635) Our Method (630,635) Fg. 5. Experments on Books. Table 4. Expermental data on Books. SIFT + ED (803,80) Our Method (803,80)

6 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp The results show 3 4 graphs n total, ncludng 4 fgures (Fg., Fg. 3, Fg. 4 and Fg. 5) and each fgure wth 4 graphs. Where (a) of each fgure s the key-feature-ponts generaton result of the mage and (b) of each fgure s the key-featureponts generaton result of the mage, the red arrows n (a) and (b) represent the key-feature-ponts; (c) of each fgure s the mage matchng result of nput mages based on our method, the cyan lnes connect the matched key-feature-ponts. At the same tme, there are 4 tables below the fgures n our experment results, where the ED n each table means Eucldean Dstance. Table -4 lst the performances of our method compared wth the method based on SIFT and Eucldean dstance. And each table ncludes column Key-feature-ponts, whch shows the number of the key-feature-ponts of the mage and mage; column Matched-Ponts, whch shows the number of the matched key-featureponts; column Matchng Rate, whch shows the matchng rate; column Matchng Tme, whch shows the matchng tme. And the Matchng Rate (MR) and Matchng Tme (MT) are defned as: Matched - ponts MR % max( Key - feature - ponts), (6) MT t + t + t 3 where t s the tme for extractng the key-featureponts of the mage, t s the tme for extractng the key-feature-ponts of the mage, t 3 s the tme for matchng. We got the Fg. 6 accordng to the tables above mentoned. In Fg. 6, the red dotted lne wth the crcle marker s SIFT+ED method results and the blue dotted lne wth the pont marker s our method results. The graph (a) shows the matchng rate of the 4 mages, and the graph (b) shows the matchng tme of the 4 mages. It s not hard to see that our method wth hgher matchng rate than SIFT+ED method from graph (a), lower matchng tme from graph (b). Analyzng the expermental results, we can fnd that the mproved mage matchng method based on cosne smlarty proposed n ths paper makes the number of correct matched ponts ncreased, corrects the error matchng, and s also more conducve to stereo matchng etc. 5. Conclusons The novel mage matchng method proposed n ths paper s based on the SIFT algorthm and cosne smlarty matchng strateg whch desgns the matchng rules based on the cosne smlartes between the feature vectors. Expermental results show that the mproved mage matchng method mproves the matchng rate effectvel makes matchng results mproved sgnfcantly. Matchng Rate /% Matchng Tme /s SIFT + ED Our Method 59.5 CONES TEDDY REINDEER BOOKS Images (a) CONES TEDDY REINDEER BOOKS Images (b) SIFT + ED Our Method Fg. 6. The graphs of expermental data. In addton, the method retans the advantages of SIFT matchng algorthm, shortens the match tme. In the long term, t may be used n the threedmensonal reconstructon and postonng n machne vson system. Acknowledgements The work s supported by Natonal Natural Scence Foundaton of Chna (No ) and the Specal Foundaton for Young Scentsts of Anhu Polytechnc Unversty (No.03RZR009). References []. G. Fu, W. Fu, H. Wu, et al., A Fast Parallel Image Matchng Algorthm Based on MPP Computng Model, n Proceedngs of the IEEE Internatonal Conference on Computer Scence and Automaton Engneerng (CSAE'), Zhangjaje, Chna, 5-7 May 0, pp []. Lowe Davd G., Object recognton from local scalenvarant feature, n Proceedngs of the IEEE Internatonal Conference on Computer Vson (ICCV'99), Kerkyra, Greece, 0-7 September 999, pp

7 Sensors & Transducers, Vol. 7, Issue 5, May 04, pp [3]. Lowe Davd G., Dstnctve mage feature from scalenvarant keyponts, Internatonal Journal of Computer Vson, Vol. 60, Issue, 004, pp [4]. Jun Luo, Yong Ma, E. Takkawa, et al., Personspecfc SIFT features for face recognton, n Proceedngs of the IEEE Internatonal Conference on Acoustcs, Speech, and Sgnal Processng (ICASSP'07), Honolulu, HI, Unted states, 5-0 Aprl 007, pp. II593-II596. [5]. Zhao Qn-Jun, Zhao Dong-Bao, We Hu, Harrs- SIFT algorthm and ts applcaton n bnocular stereo vson, Journal of Unversty of Electronc Scence and Technology of Chna, Vol. 39, Issue 4, 00, pp [6]. Sergeh Hatem Moussell Egyed-Zsgmond Elöd, Döller Maro, et al., Improvng SURF Image Matchng Usng Supervsed Learnng, n Proceedngs of the 8 th Internatonal Conference on Sgnal Image Technology and Internet Based Systems (SITIS'), Sorrento, Ital 5-9 November 0, pp [7]. A. Glnsk L. Z. Manor, SIFTpack: A Compact Representaton for Effcent SIFT Matchng, n Proceedngs of the IEEE Internatonal Conference on Computer Vson (ICCV'3), Sydne Australa, -8 December 03, pp [8]. Lu Ja, Fu Wepng, Wang Wen, et al., Image matchng based on mproved SIFT algorthm, Y Q Y Bao Xue Bao/Chnese Journal of Scentfc Instrument, Vol. 34, Issue 5, 03, pp [9]. Zeng Luan, Wang Yuan-Qn, Tan Ju-Bn, Improved algorthm for SIFT feature extracton and matchng, Guangxue Jngm Gongcheng/Optcs and Precson Engneerng, Vol. 9, Issue 6, 0, pp [0]. G. Kordelas, P. Daras, Robust SIFT-based feature matchng usng Kendall's rank correlaton measure, n Proceedngs of the Internatonal Conference on Image Processng (ICIP'09), Caro, Egypt, 7- November 009, pp []. D. Impedovo, G. Prlo, L. Sarcnella, et al., Analyss of Stablty n Statc Sgnatures usng Cosne Smlart n Proceedngs of the Internatonal Workshop on Fronters n Handwrtng Recognton (IWFHR), Bar, Ital 8-0 September 0, pp []. D. Scharsten, R. Szelsk, Hgh-accuracy stereo depth maps usng structured lght, n Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR'03), Madson, WI, 8-0 June 003, pp. /95-/0. [3]. D. Scharsten, C. Pal, Learnng condtonal random felds for stereo, n Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR'07), Mnneapols, MN, 7- June 007. [4]. H. Hrschmüller, D. Scharsten, Evaluaton of cost functons for stereo matchng, n Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR'07), Mnneapols, MN, 7- June 007. [5]. Demo Software: SIFT Keypont Detector ( 04 Copyrght, Internatonal Frequency Sensor Assocaton (IFSA) Publshng, S. L. All rghts reserved. ( 8