Image Registration and Mosaicking Based on the Criterion of Four Collinear Points Chen Jinwei 1,a*, Guo Bin b, Guo Gangxiang c

Transcription

1 nd International Conference on Mechanical, Electronic and Information Technology Engineering (ICMITE 2016) ISBN: Image Registration and Mosaicking Based on the Criterion of Four Collinear Points Chen Jinwei 1,a*, Guo Bin b, Guo Gangxiang c Institute of Metrology, Zhejiang Province, Hangzhou , China a chenjinwei_1987@126.com b urchinbin@gmail.com c guanagu@163.com Keywords: Mismatching; Cross-Ratio; Collinear; Image mosaicking Abstract. In order to avoid the influence caused by mismatching, improve the stitching accuracy, image registration based on refined feature points has been proposed. First, Harris corner detection is utilized to extract the feature points. Then the collinear points set have been selected from all of the feature points. Select the number of collinear points which are greater than or equal to 4. The collinear points from two images have been estimated by Cross-Ratio criterion. If the value of Cross-Ratio meets a certain threshold, these collinear points will be retained. 4 groups of points which are extracted from collinear points set will be used to calculate equation solution of projection transformation. Finally, fusion method combining optimal seam with feathering has been used to stitch images. An increase in the number of mismatching will cause a decline of image registration precision. There is a fall of 14.7% accuracy when one group of mismatching appears. There is a fall of 36% accuracy when three group of mismatching appears. However, the accuracy of our method has almost been unaffected. The collinear points extracted by Cross-Ratio rule are a good way to avoid the interference of mismatching. The precision can be guaranteed in the experimental process. 1. Instruction Image mosaicking is to stitch two or more images together which contain overlap area, form a wider scene image. Wide coverage and high resolution are very important in the process of the image application. The high resolution can improve the target scene detail which makes the feature information extraction and analysis easier. Larger coverage can make image covers more of the target scene, which help to study global content. In the application process, high resolution and large coverage can t be required at the same time. Image stitching technology can make use of a single sensor for many times or multiple sensor imaging for once to overcome the contradiction. Early image stitching technology is mainly using the Fourier transform method to estimate translation and rotation parameters [1, 2]. This method can only estimate simple mode l, such as translation, rotation or scaling, so it is hard to handle some complicated situation, such as affine transformation. Algorithm based on feature points now achieve excellent result, such as Scale-invariant Feature Transform(SIFT) [3] Harris-Corner [4] and Speed Up Robust Features (SURF) [5]. Besides point features, line features and regional characteristics in the field of remote sensing [6, 7] and biomedical field [8-13] have very good application prospect. Domestic and foreign scholars [14-17] use this characteristic information to greatly improve the efficiency and quality. In this paper, we choose Harris to extract feature points. 78

2 2 The criteria of Cross-Ratio Some algebraic measures structure do not change under projective transformation such as the cross ratio of four collinear points and five coplanar points which can be selected as the criterion to filtrate outliers. The cross ratio of four collinear points is chosen in this paper to select the pair of matching feature points. Figure 1. Four collinear projective invariant. The cross ratio of four collinear points is shown in figure 1. Firstly, four points of object plane ( A,B,C,D )are respectively imaged on the camera 1 and camera 2 which are represented by ( A1,B2,C1,D1)and( A2,B2,C2,D2). If the four points are on a straight line, the two groups of four points ( A1,B2,C1,D1)and ( A2,B2,C2,D2)are collinear on their respective focal plane. The reason is that the linear features and the Cross-Ratio Cr are not change in the projection transformation. d( AB) d( CD) Cr d( AC) d( BD) 3 The optimization of refined feature points The optimization of control points is mainly making use of the Cross-ratio criterion to select a little of precise matching feature points, which can avoid the interference of mismatch. Figure 2 (a) is common stitching algorithm flow chart. In the process of the global error optimization, the goal is to reduce all the registration error of feature points. If there is a certain amount of error matching, the optimization results will affect the final image stitching quality. Figure 2 (b) is the flow chart of our algorithm, which utilize the Cross - ratio rule to select a small amount of optimal matching pairs to these feature points to estimate the transformation parameters between images, and realize the optimization of image stitching. (1) (a) The flow chart of normal stitching algorithm (b) The flow chart of our method Figure 2. The flow chart of two methods. 79

3 The optimal matching pairs are the result of the rule of Cross-ratio after optimization, the principle is selecting matching pairs under projection transformation from two groups of feature points. The specific algorithm process is as follows 1) Suppose that the set of feature points is { C i } i 1,2,... n from conference image I C, we choose two points C k and C l to build linear equation. 2) We choose the feature points set { V j which can meet the equation. If the number of set { V j } is less than two, repeat the step one. If the number of set { V j } is more than two, we select the feature points T k, T l and { W j form test image I T which are Corresponding to C k, l C and { V j. 3) The equation will be built according to points T k and T l, we should judge the collinear equation between the equation and the set{ W j. If the number is less than two, repeat the step one; If the number is more than two, we calculate the cross-ratio for the straight lines respectively to obtain the values Cr( ) and Cr( ). 4) We select a threshold, If Cr( ) Cr( ), The collection of four collinear meet the requirements, otherwise the selection is failed, return to the step one. 5) Repeat this step for k times, then choose a 2 or 3 groups with minimum value. Build the sets and according to the feature points. We select four matching pairs from the sets and to calculate the transformation relationship between images. 6) Using the transformation matrix to stitch the images IC and I T. We choose the average difference of the pixel to assess the quality of stitching images. E IC( x, y) IT( x, y) /( M * N) (2) Where (, ) C ( x, y ), M * registration. I x y and I (, ) 4. Analysis of stitching T M, N x y are the gray value of the conference and test image on the coordinates N is the pixel number of overlap area, the value of E represents the stitching 4.1 Stitching experiment The goal of image mosaic is to stitch two or more images together which contain overlapping areas. In this paper, we choose two images stitching to demonstrate our new algorithm. Figure 3 are two images, (a) is conference image, (b) is the test image. 80

4 (a) Conference image (b) Test image Figure 3. Two images for stitching. Figure 4 is the stitching result of two images.(a)is the result of Harris, (b)is the result of Random Sample Consensus(RANSAC), we have found the method of RANSAC can t remove all the wrong matching. (a) Original Harris feature point matching pairs (b) Feature point matching pairs after RANSAC Figure 4. The result of feature points matching. We select the collinear feature points in the feature set which are optimized by the method of RANSC, and the number of this feature set should be larger than four. Then the Cross-Ratio of the collinear of two images are calculated and compared. If the value of Cr has meet the range of threshold, the straight lines are the result which we need. The process of result is shown in figure 5. 81

5 (a) Four collinear points of conference image (b) Four collinear points of test image Figure 5. Extraction of four collinear points. If several groups of straight lines have found to meet the criteria of cross-ratio, we can choose four pairs of feature points to obtain the transformation relationship between images. The stitching result is shown in figure 6. Figure 6. image mosaic Analysis of error The key factor of image registration is the matching accuracy of feature points. The images for stitching maybe come from different sensors, different position or different time. The wrong matching is hard to remove in the real stitching environment. These mismatching will easy affect the result of mosaicking when processing the global optimization. The method in this paper is based on the criterion of four collinear points, the advantage of which can select refined matching feature points, so the result of stitching will not be affect by the mismatching. (a) Conference image (b) Test image Figure 7. Two images for stitching. The figure 7 contains two images captured in different viewpoints. A lot of mismatching will appear in the processing of feature matching, the number of mismatching is still large even 82

6 if applied with RANSAC. The result is shown in Figure 8. Figure 8. Feature matching image. When mismatching appeared, we still can select collinear feature points which are accurately matching. When the number of mismatching is 1 to 10, the algorithm in this paper can further eliminate these false matching, the extraction results are shown in figure 9. (a) Four collinear points of conference (b) Four collinear points of test image Figure 9. Result of four collinear points when there is mismatching. In the ordinary stitching algorithm, the mismatching feature points are also the constrained optimization condition, so the registration error will expand when the number of mismatching are increasing. Because of the existence of mismatching pairs, the registration accuracy for figure 8 is vulnerable to pollution. The figure 10 is t the relationship between the matching number and registration accuracy. If we use the equation (10) for comparative analysis, the result shows that the global registration accuracy has decreased by 14.7% when there is one group of mismatching; decreased by 36% when 3 groups of mismatching, however the result of our method is almost unaffected. Figure 10. Relation between precision and the number of mismatching. 5 Summary The accuracy of registration algorithm based on feature points depends on the matching precision. With development of imaging technology, the image size and the detail resolution has been increased, the number of feature points extracted from an image is also growing. However the 83

7 increased number of feature points, the number of mismatching is also increasing. The optimization algorithm based on global feature points will be strongly affected by this mismatching. In this paper, we use the criteria of cross ratio to extract a small amount of precise feature points, which is a good way to avoid false matching caused by the interference. The result shows that the global registration accuracy has decreased by 14.7% when there is one group of mismatching; decreased by 36% when 3 groups of mismatching, however the result of our method is almost unaffected. Reference [1] Reddy B.S., Chatterji B.N. An FFT-based technique for translation, rotation, and scale-invariant image registration [J]. IEEE transactions on image processing. 1996, 5(8): [2] De Castro E., Morandi C. Registration of translated and rotated images using finite fourier transforms. [J]. IEEE transactions on pattern analysis and machine intelligence. 1987, 9(5): [3] Lowe D.G. Distinctive image features from scale-invariant keypoints[j]. International journal of computer vision. 2004, 60(2): [4] Harris C., Stephens M. A combined corner and edge detector. [C]. Manchester, UK, [5] Bay H., Tuytelaars T., Van Gool L. Surf: Speeded up robust features [M]. Springer, 2006, [6] Chen J., Feng H., Pan K., et al. An optimization method for registration and mosaicking of remote sensing images [J]. Optik-International Journal for Light and Electron Optics. 2014, 125(2): [7] Yu X. C.H., L.V. Z.H. H., Hu D. Review of Remote sensing image registration techniques [J]. Optics and Precision Engineering, 2013(11): (in Chinese). [8] Oliveira F.P., Tavares J.M.R. Medical image registration: a review [J]. Computer methods in biomechanics and biomedical engineering. 2014, 17(2): [9] Kessler M. TU-A-19A-01: Image Registration I: Deformable Image Registration, Contour Propagation and Dose Mapping: 101 and 201 [J]. Medical Physics. 2014, 41(6): 444. [10] Valsecchi A., Damas S., Santamaría J., et al. Intensity-based image registration using scatter search [J]. Artificial intelligence in medicine. 2014, 60(3): [11] Rivaz H., Karimaghaloo Z., Collins D.L. Self-similarity weighted mutual information: A new nonrigid image registration metric[j]. Medical image analysis. 2014, 18(2): [12] Onofrey J.A., Staib L.H., Papademetris X. Semi-supervised learning of nonrigid deformations for image registration[m]. Springer, 2014, [13] Zhang H.Y., Z.H. J. W., S.J.Z. Non-rigid medical image registration based on improved Demons algorithm [J]. Optics and Precision Engineering, 2007(01): (in Chinese) [14] Gao J., Kim S.J., Brown M.S. Constructing image panoramas using dual-homography warping [C]. IEEE, [15] Holtkamp D.J., Goshtasby A.A. Precision registration and mosaicking of multicamera images [J]. Geoscience and Remote Sensing, IEEE Transactions on. 2009, 47(10): [16] Qin F. Q., He X.H., CHNE W.L., et al. Super-resolution reconstruction method of image registration[j]. Optics and Precision Engineering, 2009, 47(10): (in Chinese) [17]. NIE H.B., Hou Q.Y., Zhao M., et al. IR/visible image registration based on EM iteration of log-likehood function [J]. Optics and Precision Engineering, 2011(03): (in Chinese) 84