Feature Based Registration for Panoramic Image Generation

Transcription

1 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber Feature Based Registration for Panoraic Iage Generation Kawther Abbas Sallal 1, Abdul-Mone Saleh Raha 1 Coputer Science Departent, Babylon University, Hila Babylon, 00964, Iraq Coputer Science Departent, University of Technology, Baghdad, 00964, Iraq Abstract The construction of large, high-resolution view is an active area of research in any fields like coputer vision and coputer graphics. This research is concerned with cobining ultiple iages with overlapping region to produce a panoraic view by the detection of feature points for iages with different caera otion in an efficient and fast way. Feature extraction is an iportant and critical step in panoraic view construction. This study presents techniques of corner detection, geoetric oents and rando sapling to accurately locate the iportant feature points in the iages under noise, transforation, lighting, little viewpoint changes, blurring and copression circustances. For each extracted feature in the iage, a descriptor is calculated and the atching is done based on the hoograph transforation. The results of experients conducted on iages taken by handheld caera and iages taken fro the internet. The results show that the proposed algorith is fast and efficient copared with the panoraa generation using the ost faous SIFT ethod. Keywords: Feature extraction, otion estiation, registration, panoraic iage. 1. Introduction Panoraic photography has received a growing interest. Aong its nuerous applications, such techniques ay be used to efficiently represent video sequences, in ters of copression, enhanceent, visualization. Recently have seen a growing interest in the apping and visualization of the world s cities and sights. For exaple, systes such as Google Street View and Bing Maps enable users to browse street level iagery by presenting a panoraa-based visualization of video captured at street level fro a oving vehicle [1]. Constructing a full view panoraa requires taking any regular iages in order to cover the whole viewing space. Then, these iages are aligned and coposited into coplete panoraic iages using an iage stitching algorith. Iage registration operation is very iportant to panoraa generation. Iage Registration is the process of atching two or ore iages of the scene. This requires the estiation geoetric transforations to align the iages with respect to a coon reference. Iage registration is of great iportance in all processing and analysis tasks based on the cobination of data fro sets of iages. Iage registration algoriths can be divided into two ajor categories: feature-based ethods and area-based ethods. Feature-based ethods find relevant iage features, known as control points, such as corners, point-like structures, line intersections, line ending points or highcurvature points that can be atched between two or ore iages. Once a sufficient nuber of points have been atched by correspondence on two iages, a suitable geoetric transforation can be coputed and applied to align the. Area-based ethods, also known as correlation based or teplate atching ethods, work by finding correspondences between regions of the iages without considering any features. Soe of these algoriths are based on cross correlation in the spatial or frequency doain, or on utual inforation. The correlation can be estiated locally, for exaple, for squared regions distributed over a regular lattice, or globally for the whole iage. If two iages are correlated, then the registration process continues by finding the paraeters of a geoetric transforation that axiizes cross correlation, and the iages are aligned accordingly. The iage registration process is one of the ost coplex and challenging probles of iage analysis, where the extree diversity of iages and working scenarios ake ipossible for any registration algorith to be suitable for all applications.. Related Works Many advanced ethods have been presented in recent years. [] Proposed a ethod based on the scale invariance feature transfor (i.e. SIFT algorith to stitch iages captured by the video caeras together to for panoraic Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

2 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber iages. Based on the SIFT features and the KDT structure, the best pin first searching strategy is eployed to atch feature points. Then, in a post-processing pass, the author reoves the isatching feature points. Iages captured by a surveillance caera are taken as the input to test the proposed ethod. A very popular approach is the panoraic iage osaic concept proposed by Szeliski [3] and its variants. With these approaches a rotation atrix is associated with each input iage and to reduce registration errors a local as well as a global registration step is initiated. [4] Proposed the video serial iages registration based on a feature based ethod algorith. A atching ethod based on the iproved algorith can get better iage fusion and iage registration which is introduced for attaining ore precise aggregate of atching points. In order to estiate the fundaental atrix which describes the whole geoetry accurately and robustly, an iproved SVD decoposition with weighted noralized fundaental atrix calculating ethod is proposed. In [5] the researchers using an algorith based on Correlation. For Autoatic Iage Registration Applications, the features like edges are detected by using Sobel Edge Detection Algorith. For atching the features, first Segenting the iage file in ters of different blocks and then applying the Hierarchical atching to create a pyraid of blocks. Finally, applying correlation based atching starting fro the top level of the pyraid. Otherwise take a suitable pixel block size say about 3 x 3 pixel block fro right iage and search for the exact location of that 3 x 3 pixel block in the left iage. [6] Proposes the feature based iage osaic approach. The osaic iage syste includes features point detection, feature point descriptor extraction and atching. A RANSAC algorith is applied to eliinate the nuber of isatches and obtain a transforation atrix between the iages. The input iage is transfored with the correct apping odel for iage stitching. In this paper, feature points are detected using steerable filters and Harris, and copared with traditional Harris, KLT, and FAST corner detectors. [7] Presents an approach for the panoraic view generation. First, salient features are robustly detected fro the input iages by a robust algorith called Scale Invariant Feature Transfor (SIFT. SIFT features are invariant to translation, rotation, iage scaling and partially invariant to viewpoint, illuination changes and iage noise. These features are atched between the successive iages and hence iage transforation is estiated. The iage blending technique blends the iages together to get a panoraic view without visible edge sea. Our contribution is introducing a new direction for developing the used ultiedia and devising new Media ore arousing and attracting for viewers. A new ethod for invariant feature detection is introduced based on ixing local and global feature extraction ethods. Soe researchers worked on designing the panoraic iage, but this work ais to develop an algorith to produce the panoraa in a siple and accurate way. The reainder of this paper is organized as follows. Section describes the general syste. Section 3 includes the ethodology. Section 4 explains the steps for features detection and descriptors construction. Section 5 explains the feature atching operation and otion estiation. Section 6 discusses the experiental results and section 7 explains the conclusions. 3. Methodology The panoraic iage is constructed by alignent of ultiple iages with overlapping region. The alignent operation requires detection of siilar regions in iages. The siilarities between iages are deterined by a new feature extraction ethod. The first step is capturing two or ore iages using handheld caeras. Then, the iportant features in each iage are extracted using a local feature extraction ethod. Each extracted feature needs to calculate a descriptor that deterines its invariability. The feature descriptor is calculated using a region feature extraction ethod. After that, the feature descriptors for both iages are atched using a distance etric. The atching result is refined using second atching step based on transforation technology and the otion between the two iages is estiated accordingly. The final step is using the estiated otion in the construction of the panoraic scene. The coplete steps of the proposed syste are shown in fig. 1 below. A hand held caera with a 16 ega-pixel resolution is used. Since the iages are taken by the user. To satisfy the overlapping requireent, the caera is fixed with soe distance (d and angle (Q to obtain the overlapping between the captured iages. Fig.1 The general syste Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

3 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber Features Detection and Descriptors Construction 4.1 Corner Points Detection Using Harris Detector The corner is a location in the iage where the local autocorrelation function has a distinct peak. Corner point detection has found its application in various coputer vision tasks. In this work, the Harris corner detector is proposed to extract corner inforation as the first step of the proposed algorith. The Harris corner detector was proposed by [8]. Harris Corner Detector is one of the proising tools to analyze the corner points. This ethod not only solved the proble of the discrete shifts, but also it deals with the issue of directions with the advantage of the autocorrelation function and increased the accuracy of localization. Feature point extract by the Harris vertex arithetic operator has a rotation and translation invariability and has a good robustness against noise and change of paraeters during acquisition of data. Harris detector is based on the autocorrelation function or the iage gradient [9]: x y I X I X I Y M exp (1 I X IY I Y I Where I x I, I x y y Where, I x and I y denote the iage gradients in the x and y directions. The feature point corresponds to the highest singular values, and it can be coputed using the criterion: h = DET (M k. Tr (M ( k Є [0.04, 0.06] A corner is said to occur when the two eigenvalues are large. Based on h, the pixels are classified as Corner pixel if h > 0, flat region pixel if h ~ 0 and edge pixel if h < 0. Then, set all h (x, y below a threshold T to zero. The last step in corner detection is perforing the non-axial suppression to find local axia. The flowchart of corner detection algorith and results are shown in fig.. 4. Feature Descriptor Construction Using Moents After detecting all iage features using corner detection ethod, it is necessary to identify each feature point. Therefore, an efficient and siple description will be generated for each detected feature point. For each feature, a 5*5 circular window centered on this feature is deterined. Then, the window is divided to 5*5 blocks. For each block the average of geoetrical oents is calculated. The result is a descriptor of 5*5 atrix for each feature point. Moent invariants are the ost popular and widely used shape descriptors in coputer vision derived by Hu. A -D continuous function f (x, y of the order (p+ q is defined as [10]: p q x y f ( x, y dxdy (3 For p, q = 0, 1, The uniqueness theore states that if f (x, y is piecewise continuous and has non zero values only in a finite part of x-y plane, oents of all orders exist and the oent sequence ( is uniquely deterined by f (x, y. Conversely, ( uniquely deterines f(x, y. The central oents can be expressed as [10][11][1]: Where p q ( x x ( y y f ( x, y dxdy (4 10 x and 00 y For a digital iage, Equation (3. becoes p q ( x x ( y y f ( x, y (5 x y, 0, The noralized central oents, denoted, are defined as (6 00 Where p q 1 A set of seven invariant oents can be derived fro the second and third oents: Fig. The steps of the corner point detector ( 11 (7 (8 Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

4 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber ( 3 1 (3 1 4 ( 1 ( 1 ( 3 ( [( 3( 5 (3 ( [3( ( ] 1 ( 6 4 ( [( 0 ( (9 (10 (11 (1 (13 The seven invariant oents, which are invariant to translation, scaling, irroring and rotation, coposed of the linear cobination of the second-order and third-order central oents. Because of the seven oent invariants is relatively large, and to siplify coparison, aking use of logarithic ethods. At the sae tie, taking into account the possible negative oent invariants situation, you have to take the absolute value before getting logarith. 5. Feature Matching and Motion Estiation fro Correspondences Iages can be in different situations or transforations that can be resulted during caera acquiring or by the applying iage transforation techniques. According the coplexity, these transforations are rigid, affine, piecewise affine and non-rigid. Rigid registration odels are linear and only allow for translation, rotation and scale changes without any distortion. Affine transfor is also linear and support overall distortions besides shearing and stretching. Piecewise affine and non-rigid odels are nonlinear and allow for arbitrary local and global distortions. Like these transforations ay effect on atching operation. Therefore, it is iportant to find a way to deterine the true atches. The true atches can be deterined if the points fit with a predefined odel. The atching is done by coputing the Euclidian distance between two descriptors depending on the second nearest neighbor technique as follows: DES = D (i, j / D (i, a, (14 If DES < T then (i and j is atched Where D (i, j is the distance between point (i in the first iage and point (j in the second iage and (a is the second nearest neighbor point. The points (i and (j is atched if the value of DES is lower than the predefined threshold. The isatched points can be regarded as 1 1 ( (3 ( [( ( ] 7 (3 ( [3( ( ] ] outliers, which are the data that do not confor to the odel. These outliers can rigorously disturb the estiated hoograph, and consequently should be identified. Therefore, it's iportant to deterine a set of inliers, which are the data whose distribution can be explained by soe set of odel paraeters fro the presented correspondences so that the hoograph can be estiated in an optial anner. Hoograph is a concept in the atheatical science of geoetry. A hoograph is an invertible transforation of a projective plane to a projective plane that aps straight lines to straight lines. In the coputer vision field, any two iages of the sae planar surfaces are related by a hoograph. This is very iportant in coputing the caera oveent, like rotation and translation and other transforation between two iages. In atheatical definition the hoogeneous coordinates are used, because atrix ultiplication cannot be used directly to perfor the division required by the perspective projection. ' x w x h11 h1 h13 x ' ' y x w y H h1 h h3 1 ' w h31 h3 h33 Then: x' = H. x (15 The affine hoograph is a special type of general hoograph whose last row is defined as: h 31 = h 3 = 0, h 33 = 1. The rando sapling algorith [13] is suggested in this work to be applied on the putative correspondences to deterine the inliers. This algorith was first introduced by Fischler and Bolles as a ethod to estiate the odel s paraeters in the presence of large aounts of outliers. It has been widely used in the coputer vision and iage processing for any different purposes. This algorith includes two steps that are repeated in an iterative fashion. First, a set of points is randoly selected fro the input dataset and the odel paraeters are coputed using only the eleents of this set as opposed to least squares, where the paraeters are estiated using all the data available. In the second step the algorith checks which eleents of the full data set are consistent with the odel instantiated with the paraeters estiated in the first step. The set of such eleents is called consensus set [14]. The algorith ends when the probability of finding a better consensus set is below a certain threshold. In this work four points are randoly selected fro the set of candidate atches to copute hoograph. Then, select all the pairs which agree to the hoograph. A pair (x; x, is considered to agree to a hoograph H, if Dist (H. x; x <ε, for soe threshold ε and Dist is the Euclidean distance between two points. The third step is repeating steps (1 and ( until enough pairs Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

5 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber are consistent with the coputed hoograph. The results fro this step are the group of final atching features and the otion atrix that the second iage is wrapped accordingly to generate the panoraic view. 6. Experiental Results and Discussions To evaluate the perforance of the proposed algorith, the experients done on iages taken by handheld caera and iages taken fro the internet in different situations and in coparison to the states of arts SIFT algorith. For each iage, the corners are extracted as shown in court iage in fig. 3. The value of the used threshold ust be sufficient to extract enough nuber of corners. After extracting the corner points in the two iages, the descriptors for the are created by taking a 5*5 circular window around each point, divide the window to blocks of 5*5 each and finding the average value of oent invariants for the. Then, a very efficient algorith is used for atching done by finding the second nearest neighbor. The very iportant factor here is the atching threshold because it ust be chosen to get as ore atching points as possible. The next step is eliinating the isatch points or the outliers to get on only the true atches that represent the key feature points by using a rando sapling algorith. The nuber of iterations is an iportant factor here to obtain ore atches between the iages. The result of this step is shown in fig. 4. Finally, the panoraic scene is constructed according to the estiated otion fro the previous step. The constructed panoraic view is explained in fig. 5. Other executions for the proposed ethod shown in fig. 6. Fig.4 The atching features between the two iages Fig. 5 The generated panoraic view Fig. 3 Feature extraction by corner detector (a Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

6 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber proposed ethod gives us an efficient perforance under different circustances. The nuber of corners increases with the decrease in corner threshold. The corner threshold value used in experients is 0.1. (b Fig. 6 Ipleentations of the proposed ethod In coparison with the faous SIFT ethod; sift ethod is ore coplex than the proposed ethod. The proposed algorith is accurate, soeties ore than SIFT ethod because in soe cases SIFT ethod gives less nuber of atched feature points as explained in fig. 7. The nuber of extracted feature points in the proposed ethod is 63 while in SIFT ethod is 1. The accuracy of the proposed algorith is coing fro the using of the rando sapling algorith because it follows the object's otion to estiate the atched feature points under different circustances like transforations,lighten, noise, view point changes and copression. Fig.8 The No. of the extracted features using SIFT and the proposed ethod (a Fig.9 The no. of atched features using SIFT and the proposed ethod (b Fig.7. The extracted feature correspondences. (a The SIFT ethod (b The proposed ethod Fig. 8 and Fig. 9 explain the coparison in the no. of extracted features and the no. of atches after applying SIFT ethod and the proposed iage on court iage. Soe of the transforation techniques are applied on one of the two iages used for panoraic view generation to see the difference. As shown in the figures, and in experients the 7. Conclusions Panoraic view construction is an iportant topic because it needs a fusion of iage processing, graphics and coputer vision techniques. Many researchers deal with generating panoraic iage depending on the SIFT ethod but in this work, a panoraic iage using a proposed feature extraction ethod is generated. The extracted features and constructed descriptors in the overlapping region of the iages are based on corner points, geoetrical oents and rando sapling for feature atching and otion estiation fro correspondences. The ixing of local feature extraction ethod represented by corner detector and region feature extraction ethod represented by geoetrical oents and rando sapling for feature filtering and otion estiation Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.

7 IJCSI International Journal of Coputer Science Issues, Vol. 10, Issue 6, No, Noveber gives us a fast, efficient, accurate ethod and less coplexity than the SIFT ethod. References [1] E. Zheng, R. Ragura, and P. F. George, "Efficient generation of ulti-perspective panoraas", IEEE, International Conference on 3D Iaging, Modeling, Processing, Visualization and Transission, 011. [] L. Lin and G. Nan, "Algorith for Sequence Iage Autoatic Mosaic based on SIFT Feature", WASE International Conference on Inforation Engineering, 010. [3] R. Szeliski, "Video osaics for virtual environents", IEEE Coputer Graphics and Applications, 1996, pp:. [4] W. Bing and W. Xiaoli, "Video Serial Iages Registration Based on FBM Algorith", Research Journal of Applied Sciences, Engineering and Technology 5(17, 013. [5] R. Asha, K. Asha and M. Manjunath, "Iage Mosaicing and Registration", International Journal of Coputer Science Issues, Vol. 10, Issue, No 1, March 013. [6] M.V. Subraanya and Mahesh, "Feature Based Iage Mosaic Using Steerable Filters and Harris Corner Detector", I.J. Iage, Graphics and Signal Processing, 6, 013, pp:9-15. [7] S. Prajkta, K. Krishnan, and P. Anita, "A Novel Approach for Generation of Panoraic View", (IJCSIT International Journal of Coputer Science and Inforation Technologies, Vol. (, 011. [8] C. Harris and M. Stephens, "A cobined corner and edge Detector", In Alvey Vision Conference. 1988, Pp [9] Y. Wang, Y. Chen, J. Li and B. Li, " The Harris Corner Detection Method Based on Three Scale Invariance Spaces", International Journal of Coputer Science Issues, Vol. 9, Issue 6, No, Noveber 01. [10] M.K. Hu, "Visual pattern recognition by oents Invariants", IRE Trans. Inforation Theory, Vol , Pp.: [11] R. Mohaed, Y. Haniza,, S. Puteh, Y.Ali,, S. Abdul R., R. Mohd, Y. Sazali, D. Hazri and M. Karthigayan, "Object Detection using Geoetric Invariant Moent", Aerican Journal of Applied Sciences. Vol 3 no , Pp.: [1] J. Bei and L.Chen, "Map Matching Algoriths Based on Invariant Moents", Journal of Coputational Inforation Systes Vol. 7 ( , Pp.: [13] M. Fischler and R. Bolles, "Rando saple consensus: A paradig for odel fitting with applications to iage analysis and autoated cartography", Counications of the ACM, vol. 4. No ,Pp.: [14] M. Zuliani, "Coputational Methods for Autoatic Iage Registration", PHD thesis. University of California, USA,006. Kawther Abbas Sallal received the B.S. and M.Sc. in coputer science fro University of Technology, Baghdad, Iraq in 0 and 006 respectively. She is now preparing for PHD degree in coputer science in Babylon University, Babylon, Iraq. Her interests include iage processing, coputer vision, video processing, coputer graphics, bioetrics and data security. Abdul-Mone Saleh Raha is currently a professor and head of the coputer sciences Departent, University of Technology, Baghdad, Iraq. He received his M.Sc. degree fro Brunel University, UK in 198 and his PHD degree fro Loughborough University of Technology, UK in He published ore than 100 papers in coputer science and supervised on ore than 4 PHD and 57 M.Sc. students. His research interests focus on iage processing, video processing, coputer graphics, bioetrics and coputer security. Copyright (c 013 International Journal of Coputer Science Issues. All Rights Reserved.