Automatic Registration of SAR and Optics Image Based on multi-features on Suburban areas

Transcription

1 Automatic Registration of SAR and Optics Image Based on multi-features on Suburban areas Jia Weijie 1,2 1 Institute of Photogrammetr and Remote Sensing The Chinese Academ of Mapping and Surveing Beijing, P.R.China ; 2 Geometics College Shan Dong Universit of Science and Technolog Qing Dao, P.R.China sunleaver@gmail.com Zhang Jixian 1 1 Institute of Photogrammetr and Remote Sensing The Chinese Academ of Mapping and Surveing, Beijing, P.R.China ; Yang Jinghui 1 1 Institute of Photogrammetr and Remote Sensing The Chinese Academ of Mapping and Surveing Beijing, P.R.China Abstract---Image registration, the process of estimating the optimal correspondence relation between two or more images taken at different time, from different viewpoint or b different sensors, is essential for a variet of applications in remote sensing, computer vision, pattern recognition and medicine, etc. It is a prerequisite for accomplishing high level tasks such as sensor fusion, surface reconstruction, change detection, and object recognition. In order to realize the automatic registration between Snthesize Aperture Radar(SAR) and Optical images, in this paper, an automatic registration algorithm based on different features is proposed. This method is proposed according to characteristic of image of remote sensing. Patches, edge and point features are used in our paper to regist. The evaluation of the image registration accurac proves the stratified registration algorithm based on multi-features proposed in this paper work well in automatic registration of SAR image and optical image with pixel accurac. Kewords: Image registration; Feature-based registration; SAR image; Optical image; Patches feature; edge and points features; I. INTRODUCE Image registration is fundamental process to remote sensing application. This technolog can be used in multi resource date fusion, surface reconstruction, change detection, object recognition [1]. With the ever increasing number of remote sensing image from different sensors, there are requirements to registr multi-resource images, which means multi-image product needs to be put in the same geometric reference frame. Manual image registration is well established, but this method can lead to inaccurate results, and can be slow to execute, especiall if a large number of image need to be registered. The automatic registration, which can fast and high precisel match image, becomes the hotpoint of expert stud. It solves the problem which brings b manual registration. However, because of imaging mechanism and deforming reasons are totall different between multi-source images; the problem of full automatic data registration has not been solved. That question is especiall serious when the registration between SAR image and Optics image is being done because the imaging mechanism are totall different and there are alwas man interference speckles in SAR to obstacle registration process (Dowman and Dare, 1999). Man different methods have been proposed for solving the problem of automaticall registering SAR and Optical images using feature-based matching [2,3,4]. The method of registration of high-resolution SAR Image and Optics image based on the bridging-mode Constraint can work well in automatic or semi-automatic registration but there are less tie points extracted [3]. Paul Dare and Ian Dowman propose an

2 improved model for automatic feature-based registration of SAR and SPOT images [4]. This method incorporates some features extraction and feature matching algorithms which operate together to identif common features in the multi-sensor images, from which drive a large number of tie points. However, because this model does not consider the distribution of tie points and a linear transformation function has been used to correct for non-linear distortions in the images, the accurac of registration is low. In our method, we proposed a new auto-registration methods based on multi-features. Auto-registration is processing from low to high resolution, in each step the image are registered progressivel more accurac than in the previous step. Firstl, after smooth the SAR image using Lee filter combined with Local region filter, regions are got b mean shift segmentation algorithms on a lower resolution. Regions are matched based on their contributes such as size, position and shape. According to affine transform, large differences in scale and rotation are removed in this step. Then, for more accurac registration, the quantit and qualit of the previousl located tie point are refined using an edge extraction and matching algorithm on a higher resolution. In the end, we get enough tie point pairs which equabl distribution in reference and sense image. These tie points pairs can be used in a polnomial model or a Photogrammetr model for the image registration. not change in images got at different time from different sensors. The fundamental process of our experiment is based on features on different scales, we use region extraction for the primitive registration, and then more accurac edge registration is used in higher resolution. The fundamental methodolog is illustrated in figure1. Optic image Regions extraction Edge extraction on optic image Match primitives Edge matching Tie points Transformation parameters SAR image Lee filter and Local region filter Regions extraction Edge extraction on SAR image II. PROCESS OF AUTOMATIC REGISTRATION BASED ON resampling Image registration FEATURE Figure 1 the model for automatic image registration Image registration methods are customaril categorized into two mainstreams: feature-based and area-based techniques [1]. The area-based methods, work directl on image intensities, are ver sensitive to image noise, and will become unreliable when the exist significant gre level difference and images distortion between the both images. Thus, these methods are not suitable for multi-sensor images registration, not to mention for optical and SAR images. In this paper, we use registration methods based on stead features which will A. Region Extraction The multiplicative noise and additive noise will affect definition of Snthesize Aperture Radar image, at the same time it will reduce the efficienc of SAR image region extraction. Actuall, reliabilit and robustness will increase b pre-processing the image with smoothing or speckle reduction filter. Mean filter, median filter, local filter and lee-sigma filter are used for smoothing SAR images. The result proves that combination of Lee and local filter acquires best outcome

3 which greatl reduce speckle at the same times preserve image edge information. The parameters show in tablei: TABLE I. Filter Pass Sigma Value LEE AND LOCAL REGION FILTER Sigma Multiplier Size Lee NA 3*3 Lee NA 5*5 Local Region NA 7*7 Then, an automatic threshold algorithm and homologous patch extraction [4] detects and extracts patches b scanning the image for homogeneous regions. Through this method, regions with similar intensit and space features are attached to one patch, the patches satisfied following condition [4] : (1) the patches in both images are connectedness and equalit. Equalit means that all pixels in one patch must meet some similarit rules based on gra level, texture and color. And connect means ever two pixels in the patch have the path to connect each other. (2) Two neighbor patches must have some notable difference. (3) The edge of segmented patch should be orderliness, at same time, space positioning accurac should be promised. All of that rules express in mathematic Eq.(1)(2)(3)(4): image, n S = F; (1) j= 1 j Si Sj =,( i j); (2) PS ( j ) = TURE,( j); (3) P( S S ) = FALSE,( i j). (4) i j In case of F is congregation of all the pixels in one S, P(.) is equalit. i S j are subsets of F. Mean sift segmentation algorithm [8] is used to extract patches in both images. Because onl large objects in the images are extracted as patch, images are resampled to a lower resolution before segment. That would improve compute efficienc. After man patches are got, a step should be processing is that remove those unlikel to ield successfull matches. The criterion used in that process is patch size: ver small and ver large patches are removed because the are less likel to be present real objects in the ground. B. Region Matching In this paper, we use some shape parameters to construct a cost function(cf). According to find the minimum of the function to match all the patches extracted from the sense and reference image with each other. The shape parameters contain area, perimeter length and minimum distance and maximum distance from centroid point of patch to boundar [4]. See in Eq(5) 1/2 S S a a l l L L CF = S + S a + a l + l L + L Where, S is patch area, a is the perimeter length, l is the minimum distance and L is the maximum distance. The first step of matching is calculating the value of cost function between the first patch in the sensed image with all of the patches in the reference image, find the minimum value and set the two patches as candidate matched pair. Do the same operation on all patches in the sensed image and find all of the matched pairs in both images. Next, match the patches in reference image with sensed image according to cost function as last step and find all the matched pairs. The process is repeated with the order of the image reversed. Then, the result from the two processes described above have to be combined, which means find same pairs in both processes. Compared their cost Function value with threshold, if it was less than the threshold, then that pair is considered as matched pair. If not, it will be disregard. The centroids of matched pairs are considered as tie points. These tie points are used to primitive match the images use affine transform [7].See in Eq. (6). (5)

4 X cos sinx x s Y = + sin cos (6) x + Gx (, ) = exp (7) Where s is scale transform coefficient is Convolve image f( x, ) using Gauss filter function, anti-clockwise rotate angle x, pixel and lines respectivel is offset of Our experiment has proved the usabilit of the shape parameters in the patch matching, in the future research and development, the space position parameters ma be considered. C. Accurate Registration using Edge and corner Features There are two reasons restrict the accurac of patch registration. The first reason is that the patch extraction processing in a low resolution, which will limit the accurac of image registration. The other reason is that error in four shape parameters, such as area, perimeter length, minimum distance and maximum distance will make the result of primitive registration difference with the real situation. Therefore, accurate registration based on edge and points are needed. The main streams on edge extraction and matching are as follows: Step1 edge extraction Through man researches, we find that the Cann operator has better performance on edge extraction; its position accurac and noise resistance capacit are better than other first order differential edge extract operators. Therefore, we chose Cann operator to extract edge on both images. The basic ideas of Cann are that [3] : firstl, some Gauss filter is chosen for smooth the images; then, non-maxima suppression technolog are used to deal with smoothed images. The final edge images are got after then. Supposed 2-D Gauss filter function see in Eq. (7) E E x Make, G = * f( x, ) x = G * f( x, ) (8) Ai (, j) = Ex + E where Ai (, j) f( x, ). (, i j) 2 2 (9), means edge intensit of E ai (, j) = arctan E x (10), (, i j) on image Is the normal vector of (, i j) on image f( x, ). B Cann operator, the edge point is the maximum of convolution on the edge. Defining that the pixel value of edge is 1, no edge value is 0. Step2 corner extraction on the edge image Consider about the computed efficient, the corner points are extraction in the edge image. When come to match processing, we can onl match the corners in the images, that will increase the computing speed than match ever points in the edges. Supposing that and reference image. I x and I 2 Ix IxI M = w( x, ) 2 II x I are the grads of sense image, (11) Where 1, 2 are the eigenvalues of M. Measure of corner response [6] :

5 2009 Urban Remote Sensing Joint Event x2 = a0 + a1 x + a2 + a3 x 2 + a4 x + a5 2 R = det M k (tracem ) 2 (k = 0.04 ~ 0.06), (12) Where det M = 1 2 tracem = From Eq. (12), R(corner response)depends onl on 2 = b0 + b1 x + b2 + b3 x 2 + b4 x + b5 2 (13) After transformation, chose suitable methods to resample the sense image. eigenvalues of M, and it is large for a corner. R is negative with large magnitude for an edge. R is small for a flat III. EXPERIMENT OF SAR AND O PTICAL IMAGES region. That algorithm find points with large corner response function R (R> threshold) and take the points of local maxima REGISTRATION of R. Step3 points matching Through the primitive registration, the large rotation, scale, and skew between sense image and reference image have been removed. Based on that, the points matching are done. According to last two steps, onl edges and corner points are left on the images. We designed an 8 neighbor constraint method to match the corner points on edges on the both images. A search window N * N is defined on optical edge image, at the same time, a matching widow M * M is defined on SAR edge image, the centers of widows are corner points we got in last step. Figure2. the optical image of the Copenhagen, Danmark N M = T, T decided b the primitive registration accurac. In our experience, T=10, M=3 to match the corner points with each corner points in the N windows. Firstl contrast the pixel value of corner points; if the are same, we do the next step. If the are not same, then quit. if the pixel values of the neighbor in both are same, the result is 1, if the are different, the result is 0. If the sum of result is 8, the two corner points are matching points. At last, we find all the tie points pairs in both images. Step4 polnomial transformation and resampling The result of last three processing steps is a large set of tie points pair distributed across the sense image and reference image. Second polnomial transform in Eq. (13) are used based on those tie points pairs. Figure3. the SAR image of the Copenhagen, Danmark

6 The experiment area is the connect place of urban and suburban. Our method is proposed according to characteristic of image of remote sensing in experiment area. The objects in our images alwas have different characteristics. In the area, there are Lakes and buildings and forest which has nature shapes and its internal spectrums are harmonious; the have advantage for region extraction. There are also roads which advantage for edges and corners features extraction. The sizes of images are 729*1022. See in figure3 and figure4 Though cost function, the corresponding patches pairs are found in both images. The centre points of corresponding patches are used as tie points, which can be used to compute affine transform parameter. After the primitive transform, the large scale, rotation and skew are removed in both images. The transformation accurac is about 10pixels. There are 7601 points are extracted from Optics image and points are extracted from SAR image acorrding point extraction. After 8neighbor constraint matching, there are 2177 points pairs are found in sense image and reference images. From those point pairs, we select 150 best pairs to compute as final tie point pairs. Those points are used to estimate transformation parameter. Finall the registration result is accurac. Figure4. after smooth filter SAR image(left) and SAR image(right) The combination of Lee and local filter is used for the smooth of SAR image, the result shows that it reduces the noise greatl at the same time it preserve the edges, which we can find in the figure4. The parameter of this filter can be found in Table I. SAR and optical image segmentation using mean sift algorithm Chose the some parameter to segment images, the sptial scale is 7, color scale is 6.5 and minimum region is 150. we get segmentation images on SAR and Optical images as in figure5: Figure6 Edge image of Optical image using the Cann operater Figure7 Edge image of SAR image after primitive registration Figure5 the segment of SAR image(left) and segment of optical image(right)

7 IV. CONCLUDING REMARKS Image and Vision Computing, vol.21, pp , 2003 [2]Qian Du. Automatic Registration and Mosaicking for Airborne This paper has presented an improved model for automatic multi-source image registration based on feature matching, and the method has been validated using in SAR and Optical image registration. The proposed strateg is based on a procedure where images which are approximatel aligned used patch extraction and patch matching, are registered progressivel more accuratel using edge and points matching. The tradition methods to patch matching has been followed, but there are man improvement, for example, the shape parameters are used as cost function and using feature extraction in each image to improve extraction of features from opposing image. In the edge extraction and matching procure, different thresholds are used in SAR and Optic image based on different situation. And an 8 neighbor constraint method is designed to match corner point in edge image, which is eas and quick to got match point. Contrast with one single feature methods, this approach can get more tie points and accuratel registries multi-source images. V. It is with great appreciation that we thank Prof. Anke Bellmann, Department of Computer Vision and Remote Sensing, Technical Universit of Berlin. Prof. Anke Bellmann provided us the AeS-1 airborne SAR image and the relevant optical image for the purpose of EuroSDR sensor and data fusion contest Information for mapping from SAR and optical image data. Multi-spectral Image Sequences, Photogrammetric Engineering and Remote Sensing, pp ,2008 [3] Chen Fulong, Zhang Hong, Wang Chao, Registration of high-resolution SAR image and Optics image based on the Briding-mode constraint, Remote Sensing technolog and Application, Vol21,pp , June2006 [4] P. Dare, I. Dowman. An improved model for automatic feature-based registration of SAR and SPOT image. ISPRS Journal of photogrammetr & Remote Sensing.pp: 13~ [5]Chen Shen-tie, Qian Hui, and Wu Zheng, Multiresolution Image Matching Method Based on Graph-cut and Hausdorff Distrance, Journal of Image and Graphics, Vol.13, pp , June2008 [6]Harris C, Stephens M. A combined corner and edge detector. Proceeding of the 4th Alve Vision Conference, pp: ,1988 [7] C.Pan, Z.Zhang, H.Yan, G..Wu, S.Ma. Multisource data registration based on NURBS description of contours. International Journal of Remote Sensing, vol29,pp: 569~591,2008 [8] Dorin Comaniciu, Peter Meer, Mean Shift: A Robust Approach toward Feature Space Analsis, IEEE transaction on Pattern Analsis and Machine Intelligent, Vol(24), pp: ,2002 This work is supported b National Institutes Fund for Basic Scientific Research. VI. REFERENCE [1] Barbara Zitova and Jan Flusser, Image Registration methods: A Surve,