Curvelet Fusion Of Panchromatic And SAR Satellite Imagery Using Fractional Lower Order Moments

Transcription

1 th IEEE International Conference on Advanced Video and Signal Based Surveillance Curvelet Fusion Of Panchromatic And SAR Satellite Imagery Using Fractional Lower Order Moments Odysseas A. Pappas, Alin M. Achim, David R. Bull University of Bristol Bristol, BS8 1UB UK Abstract This paper presents a novel fusion method aimed at combining panchromatic and synthetic aperture radar(sar) satellite imagery. The presented method seeks to combine the advantages of the two modalities while simultaneously minimizing the effect of artifacts inherent in SAR images. The alpha-stable distribution is used to model the curvelet decomposition coefficients of the image, as it caters for their heavy-tailed nature. Coefficients are fused using a weighted average rule with the saliency and match measures derived from the fractional lower-order moments of the alpha-stable distribution [3]. Experimental results show this method to provide high-quality results that improve the perceptive quality of the image without introducing any additional artifacts. 1. Introduction Synthetic Aperture Radar (SAR) has in recent years become a popular source of remote sensing data. It is favored over other sensor platforms as its performance is not affected by weather conditions, it can produce images during nighttime and is extremely sensitive to the geometric properties of the target [1]. SAR images however suffer from intense speckle and other artifacts such as shadowing caused by the geometrical properties of the target and the range-azimuth ambiguity. Layover artifacts appear in SAR images containing vertically exaggerated objects, such as the large buildings and skyscrapers that are often found in urban areas [2]. Fusion of SAR imagery with other types of remote sensing data, such as panchromatic images, can help minimize or eliminate the effect of said artifacts on the perceptual quality of the image. This process can also improve spatial and spectral resolution and enhance the overall contrast This work was partly supported by the Technology Strategy Board through grant TS/K005103/1 Figure 1. Flowchart for the Curvelet Transform [4]. of the image. To this effect, many fusion methods have been proposed in recent years, combining the advantages of different modalities. These methods can be distinguished into various categories, such as transform based methods (most often utilizing the wavelet transform), pixel-based and ICA methods. In the past few years wavelets have been criticized for their limited ability to sparsely model sharp discontinuities in 2-dimensional data [4] [5]. New transforms such as the ridgelet and the curvelet transform (Figure 1) have been proposed aiming to surpass wavelets in image processing by addressing this issue. Curvelets are a highly /13/$ IEEE 342

2 anisotropic multiscale architecture for the representation of 2-dimensional signals that are ideally adapted to representing objects that display curve-punctuated smoothness; that is, smoothness except for a discontinuity along a general curve with bounded curvature [5]. As they offer increased sparsity of representation, curvelets are able to represent a smooth contour with far fewer coefficients than wavelets, for the same accuracy [6]. This makes curvelets especially suitable for image processing and is an indication that many wavelet based fusion methods could benefit from being applied in the curvelet domain. In this paper we present a curvelet fusion method based on a modified version of the weighted average method. First proposed in 1993 by Burt and Kolczynski [7], the weighted average rule makes use of salience and match measures to compute a pair of weights for the fusion of each coefficient. In our proposed method, we calculate these measures using the parameters of a statistical model fitted on the curvelet coefficients. Previous work [6] has shown that the distribution of the curvelet decomposition coefficients of an image exhibits a highly non-gaussian behavior. The distribution is of a leptokurtic nature, with heavy tails. The family of symmetric alpha-stable (SαS) distributions has been shown to be a very good model for coefficients exhibiting this statistical behavior and has been successfully used to model wavelet coefficients with similar properties [8]. As only moments of order less than α (0<α<2) exist for these distributions, fractional lower order moments are used to define saliency and match measures [3]. The paper is organized as follows: In Section 2, we provide a brief overview of the mathematical properties of the alpha-stable SαS distribution. Section 3 describes the fusion algorithm while Section 4 presents results obtained using the proposed method. Finally Section 5 provides a brief conclusion of the paper. 2. Alpha-Stable Distributions The curvelet coefficients of the input images are modeled using the alpha-stable SαS distribution. Curvelet coefficients present a distribution that is symmetric in nature, as is the distribution of wavelet coefficients. The study and usage of the SαS distribution is hence limited to the symmetric case. Further information on the family of SαS distributions in general can be found in [9] [10]. Two important mathematical properties of the alphastable distributions make them suitable for various signal processing tasks. They satisfy the stability property (meaning linear combinations of jointly stable variables are also stable) and stable processes arise as limiting processes of sums of independent identically distributed random variables via the generalized central limit theorem [3]. These Figure 2. Flowchart for the Fusion Process. properties hold for both the univariate and bivariate SαS distributions. As the SαS distribution lacks a general analytic expression for its probability density function, it is best described by its characteristic function which for the univariate case is given by φ(ω) = exp(jδω γ ω α ) (1) where α is the characteristic exponent defining the shape of the distribution tails (0<α 2), δ is the location parameter and γ (γ>0) is the dispersion of the distribution. As the work is in the framework of curvelet analysis, we can assume that δ=(0,0). Two special cases of the distribution exist; for α=1 we obtain the Cauchy distribution and for α=2 the Gaussian distribution. For information pertaining to the bivariate case of the alpha-stable distribution, the reader is referred to [3]. 3. Fusion Of Curvelet Representations As mentioned, the salience and match measures are calculated using fractional lower order moments of the SαS distribution. The dispersion of the distribution (γ) in a neighborhood around the coefficient of interest is employed as a measure of saliency with the match measure derived from quantities like covariations and codifferences (covariances do not exist due to the lack of finite variance for the general SαS distribution). The algorithm uses the match measure to decide whether it will fuse the coefficients by selection or averaging - this is done by comparing it to an ad hoc threshold value which in this instance was set to The salience measure is then used to determine which of the two will be copied to the fused image with the largest weight (equal to 1 if the algorithm is in selection mode). A brief outline of the method is presented below Fusion Algorithm Pseudocode 1) Decompose image into subbands using the curvelet transform. 2) For each highpass subband pair X, Y: 343

3 a) Compute saliency measures S x and S y b) Compute match measure M = 2C xy S x 2 + S y 2 (2) Where C xy is the covariance between X, Y c) Calculate the fused coefficients using the formula Z = W x X + W y Y (3) as follows: if M is below the threshold T then W min = 0.5(1 1 M 1 T ), W max = 1 W min (4) else assign W min = 0, W max = 1. Assign W min and W max to the coefficients with minimum and maximum salience respectively by comparing S x and S y. 3) Average coefficients in lowpass residual. 4) Apply the inverse curvelet transform to obtain fused image Dispersion Estimation The Fractional Lower Order Moments (FLOMs) of a SαS random variable are given by [10]: E X p = C(p, α) γp/α for 1 < p < α (5) where C(p, α) = 2p+1 Γ( p+1 p 2 )Γ( α ) α πγ( p 2 ) (6) with Γ being the Gamma function. Using the above, one can show that [3]: γ = ( E( X p ) C(p, α) )α/p (7) E([Y E(Y )] 2 = π2 (α 2 + 2) 12α 2 (8) where Y=log X. This simplifies the process to solving (8) for α and then substituting into (4) to find γ Covariation Estimation The covariation coefficient of X 1 with X 2 can be expressed as the quantity λ x1,x 2 = [X 1, X 2 ] α [X 2, X 2 ] α (9) which does not however satisfy the symmetry criterion and is also not bounded. We will therefore use as a match measure a symmetrized and normalized version of the above given by Corr a (X 1, X 2 ) = λ x1,x 2 λ x2,x 1 = [X 1, X 2 ] α [X 2, X 1 ] α [X 1, X 1 ] α [X 2, X 2 ] α (10) We refer the reader to [3] and [11] for a more detailed and rigorous mathematical derivation of the above measures. 4. Results The results of the fusion method proposed in this paper were evaluated both visually and by using computational metrics. The usage of objective evaluation methods like the mutual information criterion requires the presence of a reference image. As no such ground truth exists, such methods were not applicable. Objective evaluation of the fusion results was therefore limited to image fusion metrics like the Piella metric proposed in [12] or the Petrovic metric [13] both of which essentially measure the edge information inherent in the input images and compare it to that carried over to the fused results. These metrics alone cannot provide a conclusive evaluation of a fused image as their connection and relevance to the perceived quality of the image is questionable. The test image set presented in this paper is part of the satellite data provided for the GRSS Data Fusion 2012 contest. In particular, it is comprised of a Quickbird panchromatic image (11th Nov. 2012) and a TerraSAR-X SAR image (5th Dec. 2012). The crop shows part of downtown San Francisco and the harbor piers. The aforementioned Piella metric was found to be more in accordance with visual evaluations of the images so was preferred as the objective metric to be used. The presented image scored with this metric, a rather high value. As mentioned, visual inspection of the images proved to be a more accurate way of accessing the performance of the algorithm when compared to metrics. Results exhibited good overall contrast in comparison to other existing methods. The test image presented here (Figures 3, 4) also shows an important feature of the algorithm the SAR image contains very intense shadowing artifacts caused by the citys skyscrapers that almost completely obscure parts of the image. The algorithm successfully fuses the highresolution spectral detail of the panchromatic image without being fooled into detecting the shadowing artifacts as the salient feature. This behavior was not exhibited by all algorithms. We include for comparison a result obtained by the Maximum Selection method (using only the highest salience coefficients), that suffered from the above mentioned problem. Additionally, the curvelet domain rule was found to not introduce any additional artifacts. The similar waveletbased algorithm presented in [3] provided good results that were significantly marred by the introduction of relatively intense ringing artifacts along the edges of salient objects. These are completely absent in the proposed curvelet-based method. 5. Conclusion This paper has proposed a novel image fusion rule in the curvelet domain, making use of the alpha-stable distribution 344

4 (a) (a) Figure 3. The Downtown San Francisco test image. TerraSAR-X SAR image (a), taken on Dec. 5th 2007 and Quickbird Panchromatic image (b), taken on Nov. 11th to model the image coefficients. Fractional Lower Order Moments have been used to determine match and salience measures for the fusion method. Results show this method to be most suitable for fusing panchromatic and synthetic aperture radar images, especially in the presence of persistent radiometric artifacts. References [1] E.T. Gormus, C. N. Canagarajah, A. M. Achim, Exploiting Spatial Domain and Wavelet Domain Cumulants for Fusion of SAR and Optical Images, Proc. of 17th IEEE International Conference on Image Processing,Hong Kong, [2] I. Elizavetin, Radiometric Artifacts on SAR Images, 10th International Scientific and Technical Conference - From Imagery to Map: Digital Photogrammetric Technologies, RACURS [3] A.M. Achim, C.N. Canagarajah, D.R. Bull, Complex Wavelet Domain Image Fusion Based on Fractional Lower Order Moments, IEEE 7th International Conference on Information Fusion [4] J.L. Starck, D.L. Donoho, E.J. Candes, Very High Quality Image Restoration by Combining Wavelets and (b) Figure 4. Image fused using the Maximum Selection method (a) and image fused using the proposed Curvelet FLOM method (b). (b) Curvelets, Wavelet Applications in Signal and Image Processing IX, Proc. SPIE 4478, [5] E.J. Candes, Ridgelets and their Derivatives: Representation of images with edges, L.L. Schumacher et al. (eds), Vanderbilt University Press, Nashville, TN, [6] L. Boubchir, J. Fadili, Multivariate Statistical Modeling of Images with the Curvelet Transform, IEEE International Conference on Information Science, Signal Processing and their Applications, [7] P.J. Burt, R.J. Kolczynski, Enhanced Image Capture Through Fusion, IEEE Fourth International Conference on Computer Vision, [8] A. Achim, P. Tsakalides, A. Bezerianos, SAR Image Denoising via Bayesian Wavelet-Shrinkage based on heavy-tailed modeling, IEE Transactions on Geoscience and Remote Sensing, [9] M. Evans, N. Hastings, B. Peacock, Statistical Distributions, Wiley Series in Probability And Statistics, [10] C.L. Nikias, M.Shao, Signal Processing with Alpha- Stable Distributions and Applications, New York: John Wiley and Sons Publishing,

5 [11] X. Ma, C.L. Nikias, Parameter Estimation and Blind Chanel Identification in Impulsive Signal Environment, IEE Trans. Signal Processing, [12] G. Piella, H. Heijmans, A New Quality Metric for Image Fusion, International Conference on Image Processing ICIP Proceedings, [13] C. Xydeas, V. Petrovic, Objective Pixel-Level Image Fusion Performance Measure, Proceedings of SPIE,