Fast Image Registration Using Pyramid Edge Images

Transcription

1 Fast Image Registratio Usig Pyramid Edge Images Kee-Baek Kim, Jog-Su Kim, Sagkeu Lee, ad Jog-Soo Choi* The Graduate School of Advaced Imagig Sciece, Multimedia ad Film, Chug-Ag Uiversity, Seoul, Korea(R.O.K *Correspodig author s address: jschoi@cau.ac.kr Abstract: Image registratio has bee widely used i may image processig-related works. However, the exitig researches have some problems: first, it is difficult to fid the accurate iformatio such as traslatio, rotatio, ad scalig factors betwee images; ad it requires high computatioal cost. To resolve these problems, this work proposes a Fourier-based image registratio usig pyramid edge images ad a simple lie fittig. Mai advatages of the proposed approach are that it ca compute the accurate iformatio at sub-pixel precisio ad ca be carried out fast for image registratio. Experimetal results show that the proposed scheme is more efficiet tha other baselie algorithms particularly for the large images such as aerial images. Therefore, the proposed algorithm ca be used as a useful tool for image registratio that requires high efficiecy i may fields icludig GIS, MRI, CT, image mosaicig, ad weather forecastig. Keywords: Image registratio; FFT; Pyramid image; Aerial image; Cay operatio; Image mosaicig.. Itroductio Image registratio is a basic task i image processig to combie two or more images that are partially overlapped. Image registratio has bee used i various fields such as geographic iformatio system (GIS, aerial, satellite, weather, uderwater images, ad etc [3-5]. May image-related works ca be separated ito three mai classes which are the correlatio-based, feature-based, ad FFT-based methods. However, they caot fid the sub-pixel accuracy iformatio for image registratio whe the full-sized images eed to be registered. I particular, FFT-based approach has bee widely used sice it is fast ad more robust tha others i image brightess chages eve though it may have some problems whe it hadles oisy images i the feature detectio stage. However, its computatio cost icreases sigificatly, ad it caot fid the accurate registratio iformatio as image size icreases [3]. To solve these problems, this work employs a pyramid-based image decompositio scheme. Specifically, first, we use the Gaussia filter to remove the oise because it causes may udesired problems i image registratio works. ext, Cay edge operator is applied o a give iput image to reduce the computatioal cost dealt with. ext, the pyramid-based image decompositio is coducted to obtai the sub-pixel accuracy iformatio ad to improve the computatio speed by usig smaller image size if ecessary. The, we estimate the registratio iformatio itegrated with the computed values from each pyramid level. It is oted that this work estimates the fudametal geometric chages such as image rotatio, scalig, ad traslatio for robust ad effective registratio. The experimet shows that the proposed algorithm saves computatioal cost with maitaiig the image quality, ad it is particularly efficiet i aerial images that cotai relatively less geometric

2 warpig problem. This paper is orgaized as follows. I Sectio, the existig works are reviewed, ad we preset a proposed scheme for image registratio i Sectio 3. Sectio 4 shows the experimetal results ad compares with other baselie algorithms. I Sectio 5 cocludes this work with some discussios.. Backgroud.. Cay Edge Detectio (a (b Cay edge detector is defied by three cocepts which are error rate, localizatio, ad respose [-]. Basic idea is to detect the zero-crossigs of the secod directioal derivative of the smoothed image i the directio of the gradiet where the gradiet magitude of the smoothed image is greater tha some threshold depedig o image statistics. Specifically, it first employs a Gaussia fuctio for the image smoothig ad uses the first derivative o it, respectively, as G( = e πσ x + y x + y σ x + y G( G( σ σ = αxe, = αye. ( x y α = πσ ext, origial image ad this fuctio are covolved i both the vertical ad horizotal directios. This operatio is described i the followig equatio [-] as, I'( = g( k, l * I( ( = g( k, l I( x k, y l k = l = Where, I is a origial image, ad g is a covolutio kerel. The o-maximal suppressio step fids the local maxima i the directio of gradiet, ad suppresses all others for miimizig false edges. The, a hysteresis threshold is applied. The hysteresis threshold uses two level threshold values which are t h ad t l (t h > t l for a high ad low levels, respectively []. The performace of Cay edge detector depeds o the parameters σ, t h, ad t l. If σ is bigger, we ca make a stroger Gaussia filter. Fig. shows the procedures of Cay edge detectio. (c (d Fig.. Filterig results of Cay edge detector : (a A Origi image, (b Gaussia filterig with σ=.5, (c o-maximal suppressio, ad (d Hystersis Thresholdig with t h =8 ad t l =64... Fourier Based Registratio I Fourier-based image registratio, the first step is to fid the traslatio iformatio, ad the the rotatio ad scalig Iformatio are computed [4-5].... Traslatio A displacemet (x, y betwee two cosecutive images is formulated by f ( = f ( x x, y y. (3 Usig a Fourier shiftig property, we get j( ωx + ω y F ( ω, ω = F ( ω, ω e. (4 (a (b (c Fig.. Traslatio : (a Iput image, (b Iput image, ad (c Cross-power spectrum ext, we ca fid the cross-power-spectrum betwee images (see Fig. [4-5] as * F ( ω, ω F ( ω, ω j( ωx + ω y = e * F ( ω, ω F ( ω, ω. (5 F = δ ( x x, y y

3 From this, we ca compute the traslatio iformatio easily.... Rotatio Similarly, to get the rotatio iformatio, we first formulate the rotatio relatio i the spatial domai [4-5] as f (. (6 = f ( xcosθ + y siθ, xcosθ + y siθ ext, we covert it to the Fourier space as F ( ω, ω. (7 = f ( ω cosθ + ω siθ, ω cosθ + ω siθ After covertig the above equatio to the polar coordiates [4-5] as M ( ρ, θ θ, (8 we ca get the rotatio iformatio θ (see Fig. 3 for the procedure...3. Scale Whe a image is scaled by a to aother image, the relatio i Fourier trasform is expressed by F ( ω, ω = F ( ω / a, ω / a. (9 a Takig the logarithm, this ca be rewritte by [4-5] F ( ω, ω = F (logω log a,logω log a. ( We ca get the scale factor a from this. ote that we igore a costat /a for simplicity. Fig.4. Flow chart of the proposed Algorithm 3.. Feature detectio I this work, to reduce the computatio cost, a pyramid image decompositio scheme is used. Let the correspodig pyramid pairs ad iput images are g, g ad I, I respectively. The, the pyramid decompositio [] is described by g ( = P ( I(, =,,...,. ( g ( = P ( I( Where, ad P meas pyramid levels ad a pyramid fuctio. It is oted that the pyramid levels are determied by the degree of feature matchig. I other words, a image is iputted to the ext level of pyramid util registratio iformatio is ot foud i feature matchig. (a (b Fig.3. Rotatio ad scalig : (a Iput image ad Log-Polar, (b Iput image ad Log-Polar. a Iput image ad its pyramid edge pairs 3. Proposed Algorithm There are three mai steps i image registratio procedures. They are a feature detectio, a feature matchig, ad the paorama image geeratio. I particular, the feature detectio i registratio related works is severely affected by brightess images betwee two give images. To solve this problem ad to reduce the computatio complexity, we use pyramid edge images i feature detectio. Moreover, to accelerate the processig speed, we collect the registratio iformatio from FFT-based approach. Fig. 4 shows the whole procedures. b Iput image ad its pyramid edge pairs Fig.5. Pyramid edge pairs To reduce more computatio cost, the edges i pyramid images are used sice pixel-by-pixel processig is geerally more complicated. Moreover, whe edge images are used i image registratio, the oise from brightess differeces is reduced [3-5]. For this purpose, we use a Cay operator. Cay operator has may merits. For example, this ca reduce the oise by a Gaussia fuctio ad it gives the ideal edges. Let pyramid edge images be c, c ad the first derivative of a Gaussia fuctio be

4 h(k,l. The, the correspodig pyramid edge images [] ca be defied by c ( = h( k, l * g (, =,,...,. ( c ( = h( k, l * g ( Fig. 5 shows the pyramid edges pairs at each level. 3.. Feature matchig Oce feature detectio is doe, a feature matchig step is coducted. I the feature matchig step, the rotatio ad scale iformatio are computed before the traslatio estimatios [4-5] Estimatio of Rotatio ad scale To obtai the rotatio iformatio, first, each pyramid layers determied by the previous feature detectio step is trasformed ito the spectrum domai. ext, the trasformed image is expressed agai i polar coordiate system as show below. Suppose that C k is a Fourier trasformed image, ad it is coverted agai ito the polar coordiate M k for k=, at the th level. The, we ca easily compute M k from Eq. (8 with takig scale iformatio ito accout [4-5] as M ( ρ, θ = M ( ρ / a, θ θ. (3 Fially, the logarithm is take for simple computatio [4-5] as M (log ρ, θ = M (log ρ log a, θ θ. (4 I this step, we ca get the scale ad rotatio agles from each pyramid layer. The fial scale ad rotatio iformatio i the origial size layer is defied by averagig. Let the scales ad rotatios be S ad R, respectively, at th level. The, the origial registratio iformatio is defied by S R = = S =, R = ( is pyramid degrees (5 (S is the origial scale, ad R is the origial rotatio Traslatio is estimated after a iput image is trasformed usig a trasformatio module ad values gaied by this step. image C by Fourier shift property [4-5] as ' * C ( ω, ω C ( ω, ω j( ϖ x + ϖ y = e (6 ' C ( ω, ω C ( ω, ω Suppose that the obtaied values i the th layer are T. The, to estimate registratio iformatio with sub-pixel accuracy for a origial image, we use the values T for computig a mappig fuctio based o χ fuctio because this approach is more robust tha the others. The lie fittig is performed to meet the followig coditios [6]. T ( X, Y <= T ( This is expressed by Y ( X = y ( x; a, b = a + bx, =,,3,...,. (7 To fid the offset b ad slope a the above Eq. (7 ca be computed [6] as, Δ SS xx ( Sx, SxxS y SxSxy SSxy S xs y a =, b =. (8 Δ Δ xi yi S, Sx, S y, i= σ i i= σ i i= σ i xi xi yi Sxx, Sxy i= σ i i= σ i Oce the a ad b are solved, we ca estimate the origial iformatio X ad Y as follows: Y = a + bx y a ( Q X Y y. (9 X = b The, we ca get the iformatio for the origial image usig the first pyramid layer, T (x,y, because T (x,y gives about the half of origial positios. 4. Experimet ad Result To evaluate the performace of the proposed approach, we use two types of images: stereo ad aerial images. Additioally, we compare the results of the proposed scheme with those of the baselie algorithms for the performace compariso Estimatio of Traslatio Let C be a Fourier image for a edge image c from a pyramid layer g which is adjusted with the computed scale ad rotatio iformatio. The, we ca estimate the traslatio iformatio betwee the give iput image C ad a adjusted iput Fig.6. Image registratio usig fixed widows

5 Fig. 7. the geeral stereo image sets (Up : a ship, Middle : a doll, Bottom : a piao. Fig. 8. Aerial image sets (Up : Aerial images, middle : Aerial images, bottom : Aerial images3. Table. The PSR ad Time (CLK for stereo sets Ship Doll Piao PSR Time(CLK PSR Time(CLK PSR Time(CLK FFT Edge-FFT Widows Feature-poit Block Proposed Table. The PSR ad Time (CLK for aerial sets Aerial images Aerial images Aerial images 3 PSR Time(CLK PSR Time(CLK PSR Time(CLK FFT Edge-FFT Widows Feature-poit F F F F Block TL TL TL TL Proposed

6 The first baselie algorithm is a basic FFT method. This method requires too much computatio ad is too difficult to fid the image features [4-5]. For the secod approach, we extract the edge images from origial images before a basic FFT method. After we get edge images, we process the whole images usig a basic FFT method []. For the third oe, we split a image ito four o-overlapped sub-images. I the feature matchig step, we obtai the required registratio iformatio from each sub-image. ote that it is performed i the frequecy domai. Mai beefit of this approach is low cost ad fast [9]. The fourth oe is based o feature-poits usig RASAC []. To fid features ad the correspodeces, we use a fast corer detectio ad zero-ormalized cross correlatio (ZCC. This method ca detect more robust features i images. However, this method requires too much computatio, ad is easily stuck i local miima. evertheless, this method usually detects the most robust features from the give images [-3]. The last approach is block-based matchig. This method relatively requires high cost eve though it gives good feature detectio results, ad its performace is more robust tha the others [4-5]. I order to evaluate the performace of the algorithms with respect to peak sigal-to-oise ratio (PSR ad speed, we use two sets of images: stereo ad aerial images. The first set icludes the stereo pairs ( ship, Doll, ad piao which are commoly used i stereo-related works. The mai reaso selectig these pairs is to test the accuracy for image traslatio. The secod oe icludes aerial images for evaluatig the accuracy of each algorithm with respect to the rotatio ad scalig. Fig. 7 ad 8 show the first ad secod sets. Table ad Table show the results of the compared algorithms i terms of PSR ad Time (CLK for the stereo sets ad aerial sets, respectively. It is oted that the used pyramid level is the 4 th degrees i the case of aerial sets, while the 3 rd level i the case of stereo sets. The reaso of the differet degree pyramids is because the proposed algorithm does ot go further whe it caot fid the registratio iformatio from each pyramid level. Additioally, cay operator parameter σ is.6. I Table, TL ad F deote too log computatio time ad o feature foud, respectively. Whe the iput images have the blurrig ad large scalig as i aerial image ad 3, ad their disparity vectors are large, the feature poit-based method fails to fid the exact correspodet poits by ZCC. Therefore, it caot compute the homography i the images. This case is defied as F i this paper. Similar situatio is happeed i the block-based approach. It takes too log time to fid the features eve though we maually adjust the block ad search widow sizes, because this approach does ot take ito accout the rotatio ad scalig. We call this case as TL. It is iterestig that the performace of proposed algorithm was more efficiet i aerial image registratios. However, it was difficult for the proposed approach to apply for the geeral stereo pairs cotaiig low feature. Fig. 9 shows a graphical result for the estimated the origial traslatio of aerial images 3 usig the values gaied from each pyramid level. Fig.9. Traslatio estimatio for a aerial image3. For the comparative results, the registered images from each baselie algorithm with oe of the stereo ad two of aerial sets are show i Fig., Fig., ad Fig., respectively. I this experimet, we verify if a iput image is a plae, our algorithm is the most efficiet tha the others. Moreover, it is more efficiet for the larger image size. 5. Coclusio This work preseted a FFT-based image registratio by usig a pyramid edge represetatio ad a lie fittig. Experimet results with several sets of data show that the proposed algorithm is more efficiet tha the other baselie oes with respect to PSR ad executive speed. However, it is oted that the proposed algorithm is ot that efficiet whe a give iput image size is small sice we use the pyramid images. I the other words, if the umber of pyramid levels is small, the it is difficult to estimate the accurate registratio iformatio because the lie fittig fuctio uses all

7 the iformatio from each pyramid levels. I cotrast, the proposed algorithm is efficiet ad fast for the large-sized images like aerial images. Therefore, our future work will focus o the fittig methods to estimate more accurate iformatio for the small sized images. We believe that the proposed algorithm ca be used as a efficiet tool for the applicatios that requires fast processig with the large sized images icludig aerial images. Fig.. Overlapped images i Aerial sets (Aerial 3.(Left Up: FFT, Bottom : Widow Right Up : Edge-FFT, Bottom : proposed. Ackowledgmets This work was supported by Korea Research Foudatio uder BK project, Seoul R&BD Program(TR86, Seoul Future Cotets Covergece (SFCC Cluster established by Seoul R&BD Program(57. Fig.. Overlapped images i stereo sets (Ship.(Left Up: FFT, Middle : Widow, Bottom : proposed Right Up : Edge-FFT, Middle : Feature-poit. Fig.. Overlapped images i Aerial sets(aerial (Left Up: FFT, Middle : Widow, Bottom : Block Right Up : Edge-FFT, Middle : Feature-poit, Bottom : proposed. Refereces [] J. Cay, A Computatioal Approach to Edge Detectio, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, vol. 8, pp , ov [] Hog Sha eoh ad Asher Hazachuk, Adaptive Edge Detectio for Real-Time Video Processig usig FPGAs, GSPx 4 Coferece, Altera, O. CF-EDG355-., May 5. [3] Barbara Zitova ad Ja Flusser, Image Registratio methods: Survey, Image ad Visio Computig, vol., Issue, pp. 977-, Oct. 3. [4] Rya Eustice, Oscar Pizarro, Haumat Sigh, ad Joatha Howlad, UWIT: Uderwater Image Toolbox for Optical Image Processig ad Mosaickig i MATLAB, Uderwater Techology,. Proceedigs of the Iteratioal Symposium o, pp. 4-45, Aug.. [5] B.S. Reddy ad B.. Chatterji, A fft-based techique for traslatio, rotatio, scale-ivariat image registratio, IEEE Trasactios o Image Processig, vol. 5, o. 8, pp. 66-7, Aug [6] Willia H.Press, SaulA. Teukolsky, Wiliam T. Vetterlig, ad Bria P. Flaery, umerical Recipes Electroic Editio Secod Editio, Cambridge, Chapter 5, 99.

8 [7] Leo J. Gleser ad David S. Moore, The Effect of Depedece o Chi-Squared ad Empiric distributio Tests of Fit, The Istitute of Mathematical Statistics, vol., o. 4, pp. -8, Dec [8] Adreas Krutz, Michael Frater, ad Thomas Sikora, Widow-Based Image registratio usig variable widow sizes, IEEE Iteratioal Coferece o Image Processig, pp , Sep. 7. [9] Lisa Gottesfeld Brow, A survey of image registratio techiques, Columbia Uiversity, Ja. 99 [] Tomas C. Hederso, E. Triedl, ad R. Witer, Edge-Based Image Registratio, Proc d Scadiavia Coferece o Image Aalysis, pp. 6-, Ju. 98. [] Richard Hartley ad Adrew Zisserma, Multiple View Geometry i computer visio d edtio, Cambridge, Chapter 4, 3. [] Edward Roste ad Tom Drummod, Machie learig for high-speed corer detectio, Europea Coferece o Computer Visio, vol., pp , May 6. [3] Edward Roste ad Tom Drummod, Fusig poits ad lies for high performace trackig, IEEE Iteratioal Coferece o Computer Visio, vol., pp. 58-5, Oct. 5. [4] C. K. Cheug ad L. M. Po, ormalized partial distortio search algorithm for block motio estimatio, IEEE Trasactios o Circuits System for Video Techology, vol., o. 3, pp. 47 4, Apr.. [5] J.. Kim, S. C. Byu, Y. H. Kim, ad B. H. Ah Fast Full Search Motio Estimatio Algorithm Usig Early Detectio of Impossible Cadidate Vectors, IEEE Trasactios o Sigal Processig, vol. 5, o. 9, pp , Sep..