Corner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity

Transcription

1 Journal of Sgnal and Informaton Processng, 013, 4, do:10.436/jsp b00 Publshed Onlne August 013 ( Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty Chn-Sheng Chen 1, Kang-Y Peng 1, Chen-Lang Huang 1, Chun-We Yeh 1 Graduate Insttute of Automaton Technology, Natonal Tape Unversty of Technology; Department of Electronc, Electrcal & Computer Engneerng Unversty of Brmngham, UK. Emal: sant@ntut.edu.tw, t @ntut.edu.tw, t @ntut.edu.tw, CXY907@bham.ac.uk Receved May, 013. ABSTRACT Ths paper presents a corner-based mage algnment algorthm based on the procedures of corner-based template matchng and geometrc parameter estmaton. Ths algorthm conssts of two stages: 1) tranng phase, and ) matchng phase. In the tranng phase, a corner detecton algorthm s used to extract the corners. These corners are then used to buld the pyramd mages. In the matchng phase, the corners are obtaned usng the same corner detecton algorthm. The smlarty measure s then determned by the dfferences of gradent vector between the corners obtaned n the template mage and the nspecton mage, respectvely. A parabolc functon s further appled to evaluate the geometrc relatonshp between the template and the nspecton mages. Results show that the corner-based template matchng outperforms the orgnal edge-based template matchng n effcency, and both of them are robust aganst non-lner lght changes. The accuracy and precson of the corner-based mage algnment are compettve to that of edge-based mage algnment under the same envronment. In practce, the proposed algorthm demonstrates ts precson, effcency and robustness n mage algnment for real world applcatons. Keywords: Corner-Based Image Algnment; Corner Detecton; Edge-Based Template Matchng; Gradent Vector 1. Introducton In automatc optcal nspecton (AOI) systems, speed, precson, and robustness are the key requrements n real world applcatons. The technque of mage algnment can be used to acheve these requrements. The common mage algnment technology can be categorzed nto two knds [1]: (1) area-based matchng, and () feature-based matchng. In the area-based template matchng, normalzed cross correlaton (NCC) method s a popular one that can be used to evaluate the degree of smlarty between template and scene mages. However, t s not robust aganst non-lner lght changes and occluded objects []. The man advantage of the feature-based mage algnment s ts effcency. Chen et al. [1,3] proposed an mage algnment algorthm based on Fourer descrptor. Ths s a knd of feature-based algnment algorthm, so-called contour-based mage algnment. Here, the Fourer descrptor s used to descrbe the contour nformaton of an object. The contour nformaton s extracted usng the maxmally stable extremal regons (MSER) method. The MSER method can be used to effectvely extract the contour nformaton n an unstable envronment. However, the contour-based algnment algorthm s based on the contour nformaton that has been strongly nfluenced by the results of segmentaton. Thus, ths algorthm s not sutable for occluded object recognton. To adapt the complcated envronment for template matchng, Steger [4] presented a smlarty measure based on the dfference of gradent drecton n edges that s robust aganst non-lnear llumnaton changes, and t can also be utlzed to recognze occluded objects. However, the computaton of ths smlarty measure s not effcent enough for real-tme applcatons. To speed up the computaton of the edge-based smlarty measure, we have presented a corner-based smlarty measure that s based on the dfference of gradent drecton n corners nstead of n edges. Ths cornerbased smlarty measure has the same characterstc wth the edge-based smlarty measure as mentoned before. Addtonally, we have further developed a corner-based template matchng algorthm based on the presented smlarty measure. A corner-based mage algnment combnes two procedures of the corner-based template matchng and geometrc parameter estmaton. In the presented corner-based template matchng, the stabltes of corners have been nfluenced by the process of corner detecton. To address the nstabltes of corners, Copyrght 013 ScRes.

2 Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty 115 Moravec [5] detected corners usng a rectangular wndow that these corners are determned accordng to the varaton of ntensty. Harrs and Stephens [6] mproved the algorthm of Moravec to reduce the nose of the mage. Smth [7] further used a crcular wndow nstead of a rectangular wndow to detect corners. Tran [8] consdered the geometrc relatonshp of corners wth ther gray values. Tran s method has been adopted n the cornerbased template matchng because of ts effcency and robustness for corner detecton.. Archtecture of the proposed mage algnment algorthm The flow chart of ths algorthm s shown n Fgure 1. The algorthm conssts of two phases, ncludng a tranng phase and a matchng phase. The corner ponts are obtaned by the ntutve corner detecton, and the gradent of corner ponts are determned usng Sobel operator. The corner-based pyramd mage technque s used to speed up the computaton n the matchng process. Here, the corner-based pyramd mage s constructed by usng the geometrc relatonshp between each corner pont. The smlarty measure used here s based on the dfference of gradent drecton of corner ponts nstead of that of edges ponts. It can therefore decrease the complexty of computaton n the matchng process. 3. Corner-Based Image Algnment Algorthm There are four parts n the corner-based mage algnment algorthm, ncludng: 1) ntutve corner detecton; ) corner-based pyramd mage; 3) smlarty measure and search strategy; 4) refnement. The detal of each part s descrbed n the followng subsectons Intutve Corner Detecton Chen [9] proposed an ntutve corner detecton, and t s much faster than Harrs corner detector. Hence, the ntutve corner pont extracton algorthm has been adopted here. Fgure shows the flow chart of the ntutve corner pont extracton algorthm. Ths method makes a crcle around p and detects all the canddate ponts. There are two crteras to check each nterestng pont p whether t s a corner or not. The two crtera are defned as: Crteron 1 I p I pdr and I p I pdr 1 1 Crteron I( p dr ) I( a) and I( p dr ) I( a1) where p dr and p dr are two dametrcally opposed pxels on the crcle pont p. dr R cos, Rsn, R beng a chosen radus and varyng n the range [0, ]. I(p) s the ntensty of center, I( p dr ) and I( p dr ) are the ntensty of a par of dametrcally opposed pxels. * Fgure 1. Archtecture of the proposed algorthm.? Fgure. Archtecture of the feature pont extracton algorthm. Crteron 1 s a frst step to detect whether the canddate pont s corner or not. If the two dametrcally opposed pxels have approxmately the same ntensty (smaller than a threshold 1 ), we wll decde that ths pont s not a corner. Crteron of the modfed ntutve corner detecton wll remove the possblty of skew edge by checkng the neghborhood of the relatve pxels. Then, an ncremental step s added to and Crteron 1 and should be checked agan. When the angle scans all the range between [0, ] and Crteron 1 and are not satsfed, the pont p wll be a corner. Copyrght 013 ScRes.

3 116 Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty 3.. Corner-Based Pyramd Image The mage pyramd technque represents the orgnal mage n the multple resolutons usng a factor of k n each level. The mage pyramd archtecture s shown n Fgure 3. In general, the basc mage pyramd s an mage array. The wdth and the heght of each level are reduced usng a factor, k, compared wth the prevous level. The pxel value f l (, j) at level l n the mage pyramd s constructed from the prevous level. To reduce the computatonal complexty, we use the mean operator to obtan the pxel values at level l. Based on the mage pyramd technque; the approprate number of levels s manly defned by the sze of the template. On the top pyramd level, the relevant features of the template should be dstngushed. Fgure 3 shows the MCU mage usng the mage pyramd technque, and the number of pyramd levels s equal to 3. In the level 3 of the mage pyramd, the features of the template s dffcult to be recognzed. Consequently, we can only set the top level up to n the mage pyramd. The feature ponts, so-called ntutve corner ponts, can be used to evaluate the smlarty measure between the template and the nspecton mages. As mentoned n Secton 3.1, ths corner detector usually obtans several postve responses that smultaneously satsfy Crteron 1 and Crteron around the corner locatons. We adopted a score computed n each response that retans the local optmum locatons as corner. The Laplacan of Gaussan (LOG) s a commonly used score that has been proven to obtan stable corners. It could be approxmated up to a scale factor by: (1) LOG p I p dr I p I p dr 0 Accordng to mage pyramd technque, the hgher level mages are blurred by usng mean operator. Thus, the locatons of the feature ponts are unstable. To mprove the robustness of the feature ponts, we have proposed a new crteron, whch s based on the geometrc relatonshp for constructng the mage pyramd structure. Ths crteron uses the feature ponts to buld the mage pyramd structure. In the corner pyramd mage (CPI), the wdth and the heght of each level are reduced usng factor kc. At level l, the pxel value pfl (, j) s obtaned from the prevous level,.e., pfl(, j) arg max( LOG( pl1 (, j))) () where pfl (, j) s obtaned usng a maxmum LOG value n each specfc regon. The sze of the regon s N N, and and j represent the locaton of the specfc regon at the prevous level l-1. Fgure 4 llustrates a smple example of the CPI structure whch has 3 levels and the specfc regon s a 3 3 mask. There are 9 regons n level 0, where 3 regons ncludng the corners ndcated as red, blue, and orange are demonstrated the process of gradent vector nhertance between two consequent levels. Here, the LOG values are used to suppress the non-maxmum value n 3 3 mask. As shown n Fgure 4, the corners n level 0, level 1 and level are p 0 ( LOG, Gx, Gy), =1 6, p1 ( LOG, Gx, Gy), =1,, 3 and p ( LOG, Gx,Gy), =1, respectvely. In ths case, the corners p 01, p 0, and p n level 0 are nherted to,, and cor- 06 p11 p1 p13 respondng to red, blue, and orange regons n level 1. And the gradent vectors of the correspondng three ponts are stored n p 11, p and 1 p13. Smlarly, the pont p1 s nherted to pont p 1 n level, and the gradent vector s also stored n. p 1 Fgure 3. The mages of the mage pyramd technque. p 1 : (50, 46,60) p 11 : (50, 46,60) p 1 : (60,60,50) p 13 : (0,40,30) p01 : (50, 46,60) p0 : (60, 60,50) p03 : (30, 6, 43) p04 : (17, 18,5) p05 : (18, 35,38) p06 : (0, 40,30) Fgure 4. The CPI archtecture, the levels of ths case s equal to. (c) Fgure 5. The results of Intutve corner detecton on conventonal mage pyramd and CPI. The orgnal mage, The CPI at level 1 (N = 3), (c) The conventonal mage pyramd at level 1. To make a comparson of the conventonal mage pyramd and the CPI, the results of ntutve corner detecton are shown n Fgure 5. As can be seen, the CPI preserves the man feature ponts n level 1 whle the Copyrght 013 ScRes.

4 Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty 117 conventonal method abandons damages the orgnal structure. The CPI s more robust and stable to the conventonal one wth the mage pyramd technque. These feature ponts wll be consdered n the determnaton of the smlarty measure between the template and the nspecton mage. The detal of the smlarty measure s descrbed n next secton Smlarty Measure and Search Strategy Smlarty measure s used to recognze the template exsted wthn the scene mage. The use of the dfference of gradent drecton shows ts robustness to lght changes. The gradent of the selected corner ponts wth the coordnate nformaton are stored as the template model whch contans a set of ponts p (, ) T x y. These ponts are relatve to the center of gravty, and the feature ponts of the template are represented usng gradents d (, ) T t u, 1,, n. The ponts and ther correspondng gradents are respectvely denoted as q (, ) T and,, n the r, c r, c ) T r c e ( v,w 1,, n canddate objects. If there s not rotaton angle between the template and the nspecton mage, the smlarty measure s defned as follows n 1 tv xr, yc uw xr, yc xy, (3) n 1 t u v w xr, yc xr, yc where n s the amount of the feature pont. In contrast, f there s a rotaton angle between the template and the search mage, the smlarty measure should be modfed by n 1 tv xr, yc uw xr, yc eg ( xy,, ) (4) n 1 t u v w xr, yc xr, yc where s a rotaton angle; the ( t, ) T u s the gradent vectors after rotaton. Perfect match corresponds to 1 n smlarty measure. The search strategy wll be the subject of a separate paper Refnement The rotaton angle s obtaned usng the neghborhood of that notes - opt, opt, opt and opt. The optmal rotaton angle s defned as opt 1 cos 1 d l (5) where l s the maxmum dstance between the corner pont and the center, d s the pxel dsplacement. The d s equal to 1 here. The above fve rotaton angles and ther correspondng smlarty coeffcents are fed nto the fttng model to obtan the refnement angle. The computaton of the best-fttng parabola s acheved by solvng a system equaton, where each rotaton angle provdes one equaton of the form, (6) The whole system can be wrtten as (7) The three unknown parameters, and, represented by vector X n the Eq. 7, are calculated from the known varables and (matrx and vector n the Eq. 7, respectvely). Takng the values of for the fve nodes descrbed above, the matrx and the vector are flled wth these fve nodes as: n n n n n n 1 n 1 n1 n 1 1, = n n 1. (8) 1 n1 n 1 1 n n 1 The followng equaton yelds the three unknown parameters, and as: T 1 T (9) After the determnaton of the parabola parameters, * and, the optmal refnement angle can be calculated by / (10) * 4. Results and Dscusson The experments are dvded nto three cases as (1) rotaton estmaton, () computaton cost and (3) robustness for comparson. Fgure 6 shows the two cases of the nspecton mages and ther correspondng template mages. The szes of two template mages are and , respectvely, and the nspecton mages are and 51 51, respectvely. These experments were performed wth Vsual C++ on Intel core GHz wth 4GB of memory Rotaton Estmaton In ths case, the nspecton mages were rotated from -0 to 0 for performance evaluaton. Fgure 7 descrbes the comparson of the corner-based mage algnment algorthm and the edge-based mage algnment algorthm wth the average error and the standard devaton error. These two quanttatve measures are used to verfy the accuracy and precson of the algorthms as lsted n Table 1. Fgure 7 further shows the errors for each rotaton Copyrght 013 ScRes.

5 118 Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty mage carefully n corner-based mage algnment. Fgure 6. The nspecton mage and template mage: Case 1: PCB mage, Case : BGA mage. 4.. Computaton Cost Fgure 8 shows the computaton cost of the two compared algnment algorthms. These fgures clearly show that our algorthm s more effcent than edge-based one n these two cases. The average computaton tmes of the proposed algorthm n Case 1 and Case are 43.3 ms and 37.5 ms, respectvely. Fgure 7. The comparson n accuracy for dfferent cases n the corner-based and edge-based algnment algorthms, respectvely: Case 1: PCB mage, Case : BGA mage. Table 1. The algnment results. Proposed mage algnment Edge-based mage algnment Case 1 Case Case 1 Case Average error ( o ) Standard devaton error ( o ) angle. As can be seen, the corner-based mage algnment outperforms the edge-based one n accuracy and precson estmaton of Case 1 and Case. Ths nterprets that the corners obtaned from the templates of Case 1 and Case are more dstnctve than the edges. Although our algorthm demonstrates that t s more accurate and precse than the edge-based one for rotaton estmaton n these two cases, t would be falure when none or only fewer corners are obtaned from the mages. Ths means that we have to select the template Fgure 8. Executng tme of case 1 and case wth the competng method. The computaton cost of the corner-based and the edged-based algorthms are both nfluenced by the amount of feature ponts. Hence, the corner ponts are used n smlarty measure calculaton and pyramd mage reconstructon nstead of usng edge ponts to speed up the computaton. Results nterpret that ths s an effectve way to smultaneously reduce the computatonal complexty and reman the accuracy for mage algnment Robustness To evaluate the robustness, the proposed algorthm was utlzed under the llumnaton changes. We used the commercal software PhotoCap to smulate the llumnant varaton. The smulated lght drectons were dvded nto 0 degree, 90 degree, 180 degree and 70 degree. The exposure value was set from 1 to for each drecton. Fgure 9 shows the llumnaton drecton wth 180 and 0 degrees. In Table, the proposed algorthm Copyrght 013 ScRes.

6 Corner-Based Image Algnment usng Pyramd Structure wth Gradent Vector Smlarty 119 shows that t s comparatve to the edge-based one n robustness evaluaton. Both of the corners and the edges were not nfluenced by the llumnaton varaton n ths experment. edge-based one. Ths algorthm especally suts to the template mage whch contans enough dstnctve corners. In the nearly future, the search strategy and the full comparson of NCC-based, edge-based and corner-based smlarty measures for mage algnment wll be presented n a separate paper. Fgure 9. Case 1: PCB mage, Case : BGA mage. Table. The mage algnment results for robust test. Drecton( o ) Exposure value Estmaton angle( o ) Estmaton angle( o ) Corner-based Edge-based Case 1 Case Case 1 Case Conclusons The man contrbuton of the corner-based mage algnment algorthm s to ntroduce the corner-based pyramd mage (CPI) and the corner-based smlarty measure. The CPI archtecture and the ntutve corner detecton are ntegrated to mprove the effcency and robustness n the matchng process. The gradent vectors of corner ponts are consdered n the corner-based smlarty measure. Results show that the proposed corner-based algorthm outperforms the edge-based one n accuracy and effcency estmaton for the two cases. The robustness of the proposed algorthm s also compettve to the REFERENCES [1] C. S. Chen, A Novel Fourer Descrptor Based Image Algnment Algorthm for Automatc Optcal Inspecton, Journal of Vsual Communcaton and Image Representaton, 009, pp do: /j.jvcr [] S. D. We and S. H. La, Fast Template Matchng Based on Normalzed Cross Correlaton wth Adaptve Multlevel Wnner Update, IEEE Transactons on Image Processng, Vol. 17, No. 11, 008, pp do: /tip [3] C. S. Chen, C. L. Huang and C. W. Yeh, An Effcent Sub-Pxel Image Algnment Algorthm Based on Fourer Descrptor, Advanced Scence Letters, Vol. 9, No. 1, 01, pp do: /asl [4] C. Steger, Smlarty Measures for Occluson, Clutter, and Illumnaton Invarant Object Recognton, Lecture Notes n Computer Scence, Vol. 191, 001, pp [5] H. P. Moravec, Toward Automatc Vsual Obstacle Avodance, Proc. Ffth of Internatonal Jont Conference on Artfcal Intellgence, Vol. 1, Cambrdge, MA, August 1977, pp.584. [6] C. Harrs and M. Stephens, A Combned Corner and Edge Detector, Proc. of 4th Alvey Vson Conference, Manchester, 31 August September 1988, pp [7] S. M. Smth and J. M. Brady, SUSAN-A New Approach to Low Level Image Processng, Internatonal Journal of Computer Vson, Vol. 3, No. 1,1997, pp do:10.103/a: [8] T. T. H. Tran and E. Marchand, Real-Tme Key ponts Matchng: Applcaton to Vsual Servong, IEEE Conference on Robotcs and Automaton, Roma, Aprl 007, pp [9] C. S. Chen, Y. H. Ku and S. H. Tsa, Fast Object Recognton Based on Corner Geometrc Relatonshp, SICE Annual Conference, Tape, 18-1 August 010, pp Copyrght 013 ScRes.