A SAR IMAGE REGISTRATION METHOD BASED ON SIFT ALGORITHM

Transcription

1 A SAR IMAGE REGISTRATION METHOD BASED ON SIFT ALGORITHM W. Lu a,b, X. Yue b,c, Y. Zhao b,c, C. Han b,c, * a College o Resources and Environment, University o Chinese Academy o Sciences, Beijing, , China b Institute o Remote Sensing and Digital Earth, Chinese Academy o Sciences, Beijing, , China - (luwd, yuexj, zhaoyh, hancm)@radi.ac.cn c Department o Airborne remote sensing, Sanya Institute o Remote Sensing, Sanya, , China Commission III, WG III/2 KEY WORDS: image registration, scale-invariant eature transorm (SIFT), Wallis iltering, adaptive smoothing, image iltering ABSTRACT: In order to improve the stability and rapidity o synthetic aperture radar (SAR) images matching, an eective was presented. Firstly, the adaptive smoothing iltering was employed or image denoising in image processing based on Wallis iltering to avoid the ollow-up noise is ampliied. Secondly, eature points were extracted by a simpliied SIFT algorithm. Finally, the exact matching o the images was achieved with these points. Compared with the existing s, it not only maintains the richness o eatures, but a- lso reduces the noise o the image. The simulation results show that the proposed algorithm can achieve better matching eect. 1. INTRODUCTION Synthetic aperture radar (SAR) is widely used in the military ield with its all-weather, all-time, strong penetration and other advantages. Side view imaging can enlarge the measuring range. SAR image registration is a key step or image processing. Because o the dierences in image mechanism, images acquired by distinct sensors perhaps appear dierent intensities and texture patterns. Especially or SAR images, noise and deormable objects o images also make image registration become more diicult. Image preprocessing is signiicant beore image analysis. Meanwhile proposing robust image registration has paid more and more attention. Generally, image registration s can be classiied into two categories, namely, intensity-based s and eature-based s. Compared with intensity-based s, eaturebased s depending on matching distinctive eatures have better eectiveness and precision. Among eature-based s, an algorithm named scaleinvariant eature transorm (SIFT) gets more and more attention in the remote sensing ields because o their distinguished perormance (Schwind et al., 2000; Lowe, 2004; Wang et al., 2012). Due to its eiciency, this is widely used in the computer vision ield to match images or to recognize and localize objects. Its invariances to scale changes, rotations, and translations and its robustness to illumination changes and aine distortion make it suitable or dierent kinds o applications such as object retrieval, image indexing, stitching, registration, or video tracking. Thereore, the SIFT algorithm and its variants are a great option or remote sensing images. Such algorithms have been applied mostly to satellite optical images since they have similar characteristics to images acquired with ordinary cameras. Several registration s use keypoints that SIFT extracts as control points to determine deormation models. Li et al. (2009) took into account the speciicity o remote sensing images and introduced a new matching criterion with scale and orientation restrictions. A multilevel SIFT matching approach is proposed by Huo et al. (2012) to register high resolution images, with the help o Random Sample Consensus (RANSAC). Sedaghat et al. (2011) adapted the algorithm to obtain key points that are uniormly distributed in space and ilter the mismatches by applying a projective model. The SIFT algorithm also has some assets or the retrieval o remote sensing images or or classiication applications. Zhang et al.(2011) proposed a SAR image registration based on the divided regions; Xia (2014) combined the corner match with prior inormation. Object detection is another application ield o the SIFT algorithm. Single buildings are detected on very high resolution optical images by Sirmacek and Unsalan (2009) by using SIFT key points, multiple subgraph matching, and graph cut s. Tao et al. (2011) perormed airport detection by considering both clustered SIFT key points and region segmentation. Although the SIFT algorithm has proved its eiciency in the application o optical remote sensing, the situation or SAR images is dierent. The SAR image is corrupted by strong multiplicative noise, which is known as speckle, and data processing is diicult. Especially, the SIFT algorithm does not word well on this type o image. Thereore, we are supposed to take into account the potential application o SAR images in the local context, it is important to adapt to this new. Several improvements have been proposed to apply the algorithm to more cases. Some suggest making the images preilter or denoise to reduce the inluence o speckle noise. Others remove some invariances, or modiy some steps o the algorithm, to improve the perormances. Spatial relationships between key points are considered by Lv et al. (2011) and Fan et al. (2013) to suppress alse matches. Perormances o these newly proposed algorithms are still relatively insuicient, and the number o correct matches is not suicient to consider other applications than image registration. However, many o them do not consider the statistical speciicities o speckle noise. * Corresponding author Authors CC BY 4.0 License. 623

2 In this paper, we propose a new algorithm improved based on the SIFT algorithm beore. Firstly, the adaptive smoothing ilter is conducted on SAR image pairs, which is an adaptive smoothing algorithm based on image gradient. According to the mutation characteristics o gray value o image pixels, we adaptively change the ilter weights and sharpen the image edge during the process o smoothing, which better eliminate the contradiction between iltering noise smoothing and sharpening the edge, with great perormance on iltering and easy real-time processing. Secondly, the images were iltered through Wallis ilter, which utilizes the Gauss smoothing operator in the calculation o mean and variance o local gray value. Moreover, it enhances the useul inormation o image, suppresses noise, increases the image signal-to-noise ratio, extracts the extremely uzzy texture inormation and eectively improves the precision. Finally, the SIFT algorithm is conducted to extract the eature points that are used to match the images accurately. The speciic process is described as ollows: 1) the scale-invariant eature points in the image are extracted; 2) the detected eature points are described, and the eature descriptors are ormed; 3) the original image and the SIFT descriptor o the target image are matched. Comparing with the existing s, our one not only keeps the richness o image eatures, but eectively reduces image noise with great accuracy o eature point detection and image matching. Through the experiment o three groups o SAR images, we adjust the parameters to achieve the best results with many trial-and-error tests. The experimental results show that the proposed algorithm can achieve good matching results. 2.1 Wallis iltering 2. PROPOSED METHOD Wallis iltering is a kind o local image transormation, the aim o Wallis ilter is to map gray scale mean and variance o image into given gray scale mean and variance. Through this iltering, the contrast o the image becomes more uniorm. Because this ilter uses a smooth operator in computing local gray scale mean and variance, it can enhance much useul inormation o image and weaken noise to some extent, which increases the SNR (signal to noise ratio) o image. The general ormula o Wallis ilter is: ( x, y) g( x, y) r1r 0 cs (1) r1 ( csg (1 c) s ) r0 bm (1 b r1) mg Where r1 is a multiplicative parameter and r0 is an additive parameter. I r1 >1, transormation (1) becomes a high pass ilter; else i r1<1, transormation (1) is a low pass ilter. mg expresses gray scale mean o image in a certain area o a pixel and can be regarded as the DC component o image signal. sg can be called the AC component o image signal, which means gray scale variance o image in a certain area o a pixel. g(x,y) and (x,y) are original image and the iltered image. m is a target value o gray scale mean, and it must be a median o dynamic image range. s is a target value o gray scale variance, and it determines the contrast o image. c is a contrast expansion constant o image, c[0,1], it should increase with the window increasing in the process. b is an image intensity coeicient. When b 1, the mean o image is orced to m, and when b 0, the mean o image is compelled to mg. Thereore, in order to maintain the gray mean o the original image, b should be used the smaller value. Multiplicative coeicient r1 decides the perormance o this ilter. And its relationship with others parameter can be rewritten in the ollowing: 1 1 sg 2 + (2) r c s To the typical Wallis ilter, c=l, b=l, hence: s ( x, y) [ g( x, y) mg] m s 2.2 Adaptive smoothing iltering 1 In order to avoid the use o Wallis ilter to enhance the image, and increase the noise inormation on the image, this paper uses adaptive smoothing ilter to denoise the image beore Wallis iltering. Saint-Marc et al.(1991) proposed adaptive smoothing iltering, which is used to smooth the image while still retaining the image discontinuities. The basic idea o this is using a locally weighted iterative convolution template and the original image signal, the process has the properties o anisotropic diusion, the weighted coeicient o each pixel in each iteration is changed, it is the gradient unction o the pixel, relect the continuity o image pixel gray value. Adaptive smoothing ilter is an iterative process, the maximum number o iterations is k, the speciic steps o the algorithm are as ollows: 1) Initial iteration times n = 0, set the maximum number o iterations k and the value o the parameter h; ( n 2) Compute the gradient o the n iteration: G ) x ( x, y) and ( G n) ( x, y ) : y ( n) 1 ( n) ( n) Gx ( x, y) ( I ( x 1, y) I ( x 1, y)) 2 ( n) 1 ( n) ( n) Gy ( x, y) ( I ( x 1, y) I ( x 1, y)) 2 3) Compute the window weight coeicients o the n ( n iteration: ) ( xy, ) g (3) (4) ( n) 2 ( n) 2 [ G (, )] [ (, )] ( ) x x y Gy x y n ( xy, ) exp( ) (5) 2 2h 4) Compute the convolution: I ( n1) ( x, y) 1 1 ( n) I ( n) x i y j x i y j i1 j1 1 1 ( n) (, ) (, ) i1 j1 ( x i, y j) 5) Whether n = k, i n =k, Stop the iteration; otherwise n = n + 1, go to step 2). The weight coeicient W is obtained by the image gradient and the parameter h, and the parameter H value determines the degree o the edge detail inormation in the image smoothing process. h larger, the edge o the details will be smooth out; h smaller, the edge and the noise are both retained, no smoothing eect. Thereore, the h value plays a decisive role in the inal image smoothing eect. Parameter h usually takes 1.5 to 2 times the standard deviation. 2.3 SIFT algorithm Through a Gaussian dierential unction, the SIFT algorithm generates a scale space, and selects local extreme point as the (6) Authors CC BY 4.0 License. 624

3 candidate eature point, then removes the unstable low contrast and edge response points, and precisely positions eature points. Image data is sampled down, and the image pyramid is constructed. Gaussian convolution kernel can achieve the scale transormation. Two-dimensional image can be ormed as image scale space with the convolution to dierent kernel Gaussian ilters. As ollows: L( x, y, ) G( x, y, ) I( x, y) (7) ( x y )/2 G( x, y, ) e (8) 2 2 D( x, y, ) ( G( x, y, k) G( x, y)) I( x, y) (9) L( x, y, k) L( x, y, ) Wherein, I (x, y) is an input image, G is the Gaussian kernel unction, σ is a variable kernel. Extreme point o Gaussian Laplace can provide stable characteristic or the image. Gaussian Laplace unction can be approximated by Gaussian dierential unction. And the extreme points can be obtained by Gaussian dierential unction. Gaussian unction its to the three-dimensional quadratic unction to achieve the positioning accuracy o the eature points. Gaussian dierential unction can be expanded by Taylor ormula on candidate extreme points. Detecting extreme point o dierenced Gaussian has a strong edge response. There is a proportional relationship between the principal curvatures o the edge points and its eigenvalue o the Hessian matrix. Calculate the eigenvalue o the Hessian matrix to determine whether it is the stable point. Lowe proposed the to construct eature points descriptors using the gradient distribution o the eature points neighbor pixels. Gradient magnitude and direction o pixel (x, y) are expressed as ollows: m (x, y) and θ(x, y) represent the magnitude and orientation o the gradient, respectively. L is the scale o each key point. m( x, y) ( L( x 1, y) L( x 1, y)) ( L( x, y 1) L( x, y 1)) L( x 1, y) L( x 1, y) ( xy, ) arctan L ( x, y 1) L ( x, y 1) 2 2 (10) (11) The gradient distribution is counted by the center pixel o the eature points. Histogram about the gradient direction is divided into 36 columns, and the peak value indicates as the main direction o the eature points. Rotate the axis o the image to the main direction o the eature points. Use Gaussian weighted principal to calculate the accumulated value o the pixel in the window in each direction o the gradient. Neighbor window is divided into 16 sub-blocks o 4 4, and statistic the accumulated value o gradient histogram o 8 directions in each sub-block, then generate a seed point. The inal descriptor is a vector holding 128 elements. Euclidean distance between eature descriptors is the similarity measure used to match two eatures. The two closest eatures in the input image are determined or every reerence image eature. I the Euclidean distance o the second-closest match is smaller than the threshold (0.6) times the distance o the closest match, the input eature is accepted as a match. Since comparing the distance o all eatures with each other would be too expensive, the approximate nearest neighbor best-bin-irst (BBF) algorithm is used. 3. EXPERIMENTS AND RESULTS This paper proposes an image registration algorithm based on SIFT, this algorithm uses adaptive iltering and Wallis iltering to process the image beore extracting the eature. Adaptive ilter is designed to eliminate noise, Wallis iltering is to increase the image texture inormation. Speciic steps are as ollows: 1) The image is denoised by adaptive smoothing iltering,detailed procedures are shown in section ) The image is divided into a ixed size, nonoverlapping rectangular window area, the scale o the window should correspond to the scale o the texture pattern to be enhanced, and the mg and sg o the window are calculated. 3) According to the parameters that input, the multiplicative coeicient r1 and the additive coeicient r0 o each region Wallis ilter are calculated. the parameters m and s values are set to 127 and the value rom 40 to 70. The parameter s should be reduced with the decrease o the regional scale, so as to avoid the saturation o the gray value o a large number o pixels (beyond [0,255]). The range o image contrast expansion constant value c is [0,1]. In this paper, the value [0.75,1] can be used to prevent the low gray value o image that is enhanced. Image brightness coeicient b ranges [0,1], this paper uses the value ranges [0.5,1] that this paper used can maintain the original image o the local gray mean. 4) According to the ormula (1), r1 and r0 o each region are used to calculate the gray value o the Wallis transorm o the central point o the region, and we can remember them as matrix R, Mean value and standard deviation o gray value ater the transormation o all regions are calculated and recorded as mr and sr. 5) The original image g(x,y) is subjected to Wallis iltering transorm by pixel. Since each window does not overlap, the coeicients o any pixel on image are obtained by bilinear interpolation and the new gray value o all pixels is calculated according to equation (1). 6) According to equation (3), the image that was iltered with Wallis will be subjected to a global classical Wallis ilter transorm again. Where mg and sg are replaced by mr and sr. The process can urther enhance the degree o image sharpening and increase the image contrast. 7) Extract the SIFT eature points and match the images that were iltered, output the images. Wallis iltering is applied to the SIFT matching process. The experiment uses three pairs o images or eature extraction, which contains a representative o the surace eatures inormation, so that the experiment has a good persuasion. The image size used in this experiment is 200 size 200*200 pixels. Firstly, the improved Wallis ilter is used to enhance the image. Wallis ilter parameters are set to: Rectangular area size 39*39 pixels,m and s were 60 and 127 respectively, b and c were 0.6 and 0.75 respectively. Then, the enhanced image is matched by SIFT, and the results are compared with the original SIFT eature extraction. In order to veriy the eect o the adaptive smoothing ilter in the Wallis iltering process, two SAR images were selected to perorm the adaptive smooth iltering denoising experiment. During the experiment, the k was 4, and the h was about 1.75 times the standard deviation. Figure 1 is the original image and its corresponding adaptive smooth iltered image. We objectively evaluate the iltering eect using PSNR (peak Authors CC BY 4.0 License. 625

4 signal noise ratio), NMSE (normalized mean square error) and SSIM (structural similarity), the statistical results shown in Table 1. As it shown that the smaller the ENMS, the better the iltering eect, the bigger the RPSN, the better the iltering eect, and the closer the SSIM to 1, the better the image structure is. The experimental results show that the adaptive iltering algorithm not only can ilter out the noise, but also protect the edge o the image, whether it is rom the visual eect o the iltered image or the image quantitative evaluation index ater denoising, is conducive to the Wallis ilter processing. (b) Diagram o SIFT matching points connection ater Wallis iltering (a) Data 1 (c) Diagram o SIFT matching points connection ater iltering twice Figure 2. The matching result o SIFT algorithm to the enhanced image (b) Data 2 Figure 1. Original image with adaptive smoothing iltered image image ENMS RPSN SSIM Data Data Table 1. image iltering results evaluation statistics It can be seen rom the experimental results that the original image is enhanced and the region o the original image with a small change in gray scale is signiicantly enhanced by Wallis iltering. The dierence between the two algorithms is not clearly dierentiated by visualization, but the results o eature extraction with SIFT are quite dierent. The number o matching points is 50, 45, and 102. Some points on the edge line have not been extracted, because the contrast on the original image is weak. However, the number o eature points matched on the image ater Wallis iltering enhancement is the least, because the image has not been denoised, the contrast and noise are magniied at the same time. It is better to extract the eature points on the image ater two iltering enhancement, corner and edge points are well extracted. The other two sets o original images and matching results are shown in Figures 3. Image enhancement processing and eature extraction results are shown in Figure 2. Figure 2 (a) shows the matching results or the original image. The original image that the gray scale is uniorm, but part o the edge area contrast is not obvious. The images that used the Wallis iltering and Wallis iltering combined with adaptive iltering to enhance are shown in (b) and (c) respectively. (a) Original image, data 2 (a) Diagram o SIFT matching points connection o original image (b) Diagram o SIFT matching points connection ater iltering twice Authors CC BY 4.0 License. 626

5 ACKNOWLEDGEMENTS The author would like to thank the anonymous reviewers or their valuable comments and suggestions. This work was supported by the key research and development plan o Hainan province under Grant NO. SY16ZY (c) Original image, data 3 REFERENCES Schwind, P., Suri, S., Reinartz, P., and Siebert, A., Applicability o the SIFT operator to geometric SAR image registration. Int. J. Remote Sens., vol. 31, no. 8, pp Lowe, D. G., Distinctive image eatures rom scaleinvariant key points. Int. J. Comput. Vis., vol. 60, no. 2, pp (d) Diagram o SIFT matching points connection ater iltering twice Figure 3. Original image and the matching result o SIFT algorithm to the enhanced image Finally, the RANSAC (Random Sample Consensus) algorithm is used to ilter the point pairs o SIFT matching and remove the wrong matching points so that the correct matching rate is obtained. The comparison o SIFT algorithm and the proposed algorithm is shown in Table 2. The results show that the SIFT algorithm can eectively extract the stable matching point, but its point is less and the correct rate is not very high. While using Wallis iltering combined with adaptive smoothing iltering beore the SIFT match, the number o point pairs and the correct rate has been signiicantly improved. matching points correct points correct rate (%) No. SIFT/Proposed SIFT/Proposed SIFT/Proposed 1 50/110 45/108 88/ / /319 89/ /305 71/278 86/91 Table 1. Result o traditional SIFT algorithm and proposed algorithm 4. CONCLUSION Under the condition o serious speckle noise and similar texture structure in SAR images, the based on SIFT eatures can still be matched to a large number o stable points. This paper selects 3 groups o dierent categories o SAR image data and uses SIFT algorithm combined with adaptive smoothing iltering and Wallis iltering to match the images, and then the RANSAC algorithm is used to eliminate the error o the matching points, in order to improve the accuracy o matching points. The experimental result shows that the SAR image SIFT algorithm can match accurately to the stable characteristics and has certain robustness. Together with Wallis iltering and adaptive iltering algorithm, it can eectively cut down the image noise and enhance the image texture, and the eective number o matching points and correct rat are increased, so this can be applied to the subsequent processing o SAR image data. Wang, S., You, H., and Fu, K., BFSIFT: A novel to ind eature matches or SAR image registration. IEEE Geosci. Remote Sens. Lett., vol. 9, no. 4, pp Li, Q., Wang, G., Liu, J., and Chen, S., Robust scaleinvariant eature matching or remote sensing image registration. IEEE. Geosci. Remote Sens Lett., vol. 6, no. 2, pp Huo, C., Pan, C., Huo, L., and Zhou, Z., Multilevel SIFT matching or large-size VHR image registration IEEE Geosci. Remote Sens. Lett., vol. 9, no. 2, pp Sedaghat, A., Mokhtarzade, M., and Ebadi, H., Uniorm robust scale invariant eature matching or optical remote sensing images. IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp Zhang, B., Tian, Z., Yan, W., An SAR Image Registration Algorithm Based on Segmentation-derived Regions. Chinses journal o engineering mathematics. 28(1), pp Xia, G., Research on image sequence alignment or radar o WAS SAR-GMTI large angle o view. Xi'an Electronic and Science University. Sirmacek, B. and Unsalan, C., Urban-area and building detection using SIFT keypoints and graph theory. IEEE Trans. Geosci. Remote Sens., vol. 47, no. 4, pp Tao, C., Tan, Y., Cai, H., and Tian, J., Airport detection rom large IKONOS images using clustered SIFT keypoints and region inormation. IEEE Geosci. Remote Sens. Lett., vol. 8, no. 1, pp Lv, W., Yu, W., Wang, J., and Wang, K., SAR image matching based on SIFT keypoints and multi-subregions inormation. in Int. Asia-Pac. Con. Synthetic Aperture Radar, pp Fan, B., Huo, C., Pan, C., and Kong, Q., Registration o optical and SAR satellite images by exploring the spatial relationship o the improved SIFT. IEEE Geosci. Remote Sens. Lett., vol. 10, no. 4, pp Saint-Marc, P., Chen, J.S., and Medioni, G Adaptive smoothing: a general tool or early vision. IEEE Trans. Pattern Analysis and Machine Intelligence. 13(6): pp Revised June Authors CC BY 4.0 License. 627