Detecting Ship Targets in Spaceborne Infrared Image based on Modeling Radiation Anomalies

Size: px

Start display at page:

Download "Detecting Ship Targets in Spaceborne Infrared Image based on Modeling Radiation Anomalies"

Morgan Higgins
5 years ago
Views:

1 Detecting Ship Targets in Spaceborne Infrared Image based on Modeling Radiation Anomalies Haibo Wang a,b, Zhengxia Zou c,, Zhenwei Shi c,, Bo Li a a Beijing Key Laboratory of Digital Media, School of Computer Science and Engineering, Beihang University, Beijing , PR China b China Centre for Resources Satellite Data and Application, Beijing , PR China c Image Processing Center, School of Astronautics, Beihang University, Beijing , PR China Abstract Using infrared imaging sensors to detect ship target in the ocean environment has many advantages compared to other sensor modalities, such as better thermal sensitivity and all-weather detection capability. We propose a new ship detection method by modeling radiation anomalies for spaceborne infrared image. The proposed method can be decomposed into two stages, where in the first stage, a test infrared image is densely divided into a set of image patches and the radiation anomaly of each patch is estimated by a Gaussian Mixture Model (GMM), and thereby target candidates are obtained from anomaly image patches. In the second stage, target candidates are further checked by a more discriminative criterion to obtain the final detection result. The main innovation of the proposed method is inspired by the biological mechanism that human eyes are sensitive to the unusual and anomalous patches among complex background. The experimental result on short wavelength infrared band (1.560µm 2.300µm) and long wavelength infrared band (10.30µm 12.50µm) of Landsat-8 satellite shows the proposed method achieves a desired ship detection accuracy with higher recall The work was supported by the National Natural Science Foundation of China under the Grants and the Beijing Natural Science Foundation under the Grant (Corresponding author: Zhengxia Zou and Zhenwei Shi.) Corresponding author: Image Processing Center, School of Astronautics, Beihang University, Beijing , PR China. Tel: ; Fax: addresses: zhengxiazou@buaa.edu.cn (Zhengxia Zou), shizhenwei@buaa.edu.cn (Zhenwei Shi) Preprint submitted to Elsevier May 16, 2017

2 than other classical ship detection methods. Keywords: Ship detection, Infrared spaceborne image, Radiation Anomaly, Gaussian Mixture Model 1. Introduction Ship detection with spaceborne remote sensing image has long been a hot issue in the remote sensing image processing field due to both its civil and military use. While most of the earlier researches for ship detection are designed for synthetic aperture radar image and high resolution optical image, with the rapid development of infrared sensor technology, it is now possible to explore ship detection method with spaceborne infrared image. Detecting ship target with infrared image has many advantages over other sensor modalities, such as better thermal sensitivity and all-weather detection capability. For the most of the previous ship detection methods, a coarse-to-fine detection strategy is usually used. Their algorithm flow can be divided into the following two stages: 1) ship candidate extraction and 2) ship verification. In the first stage, all the candidate regions that possibly contain a ship target are extracted. Some classical image processing methods like image threshhold segmentation based methods [Xia et al. 2011, Chen et al. 2011, Jubelin et al. 2014, Bi et al. 2012, Qi et al. 2015, Zhu et al. 2010] and the saliency detection based methods [Qi et al. 2015, Ding et al. 2012] are commonly used in this stage. In the second stage, a more discriminative algorithm will further check each of the individual candidate to verify whether it really contains a ship target. Previous works of this stage [Xia et al. 2011, Bi et al. 2012, Zhu et al. 2010, Tang et al. 2015, Yu et al. 2014, Zou et al. 2016] mainly transform this operation into a feature extraction and a binary classification process (targets and backgrounds). In this paper, a new detection strategy is studied compared to the previous ones. Our method is inspired by the biological mechanism that human eyes are sensitive to the unusual and anomalous patterns in an image [Tsotsos et al. 1995, Itti et al. 1998]. We assume that any image patterns in an infrared image can be decompensated as a group of similar or dissimilar patches. On one hand, some of these patches share a high degree of similarity and have a vast number of quantities. These patches build up the background region of an image. On the other hand, some patches show strong rarity and 2

3 specificity thus we call these patches radiation anomalies. In most cases, anomalous patches can be a reflection of some specific edge and textures, where these patterns cannot be well explained by their surroundings. We further assume that a ship candidate region shows a high degree of radiation anomaly in an infrared image and usually appears as an outlier in the probability distribution of the image data. To evaluate the anomaly degree of the image and better exploit their nonlinear statistical characteristics, a Gaussian Mixture Model (GMM) based evaluation method is proposed as an effective ship candidate detection method. The rest of this paper is organized as follows. In section 2 and 3, we will introduce the radiation anomaly evaluation method and details of our detection framework. Some experimental results will be given in section 4, and the conclusions are drawn in Section Proposed Method The proposed candidate detection method can be summarized into two steps: 1) feature representation; 2) radiation anomaly evaluation. Fig.1 illustrates the detection pipeline. The details of each step are given as follows. 1). feature representation. Suppose there is an infrared image with the size of H W pixels and D bands. We first densely divide this image into a N overlapping patches with p p patch-size and s s patch-stride. Since the raw pixel value of an infrared image reflects the radiation characteristic of the ground elements while the shape of a ship target can be well described by the gradients or edge directions, here we simply take a modified approach of a classical feature extraction method, the Histograms of Oriented Gradients (HOG) [Dalal et al. 2015] as a description of the image data. Concretely, we still follow a traditional processing flow of HOG where each individual image patch is further divided into M uniform spatial cell regions and for each cell, a histogram of gradient directions over 9 orientation bins which is evenly spaced over is accumulated. To reduce the interference of high frequency noise in the image, each image patch is convoluted with a 3 3 Gaussian mask before accumulating the feature histograms. For simplicity, we do not take the block-wise normalization strategy in the original HOG algorithm. The final feature description of a image patch is connected with the histograms h and their raw pixel values r of each cell in an end-to-end 3

4 Fig. 1: Framework of the proposed method: feature representation and radiation anomaly evaluation. The original feature has been projected to a two-dimension plane by principle component analysis for better illustration. 4

5 manner x = [h 1, r 1,... h M, r M ] T R L 1. (1) where L is the final dimension of the feature space. 2.) radiation anomaly evaluation. We use a GMM to evaluate the radiation anomaly of a patch extracted from a test image. A GMM can be basically formulated as p(x) = K ϕ k N (µ k, Σ k ), k=1 K ϕ k = 1 (2) where the kth component is characterized by normal distributions with weights ϕ k, means vector µ k and covariance matrices Σ k. Since ship targets usually show high degree of anomaly and rarity and are distributed as the outliers (with low probability density) of any mixture Gaussian clusters, while those background patches share high degree of similarity and are usually distributed near the cluster center (with high probability density), we define the anomaly of an image patch x based on the probabilistic density: k=1 A(x) = 1 ˆp(x) = 1 K ϕ k N ( ˆµ k, ˆΣ k ) (3) i=k where ˆµ k and ˆΣ k represent the estimation of the means and covariance matrices. After anomaly evaluation, a patch that possibly contains a target will get a large response, while a background patch will get a small one. The final anomaly score of pixel location (x, y) of a test image can be obtained by simply averaging anomaly score of the overlapped patches covered on this position. The score map of the test image can be compared with a threshold T (0, 1) and the final ship candidate region refers to those pixels with their responses larger than the threshold. To determine the parameters ϕ k, ˆµ k and ˆΣ k with an a priori given number of components K, we use the Expectation Maximization (EM) algorithm [Bishop. 2006], which is a particular way of implementing maximum likelihood estimation for such problem. EM gives iterative estimation of these parameters where a closed-form solution is possible at each iteration. Detailed iteration process is given as follows: E step: Evaluate the responsibilities using the current parameter values γ(z nk ) = ϕ k N (x n µ k, Σ k ) K j=1 ϕ jn (x n µ j, Σ j ) (4) 5

6 M step. Re-estimate the parameters using the current responsibilities µ k 1 N γ(z nk )x n N k Σ k 1 N k n=1 N n=1 ϕ k N k N, where N k = γ(z nk )(x n µ k )(x n µ k ) T N γ(z nk ). n=1 (5) The above iteration should be stopped when the posterior probability distribution convergence to a constant. Afterwards in the ship verification stage, each individual ship candidate will be further checked to verify whether if it really contains a target. Here we follow a classical processing strategy that converting this process to a binary classification operations which discriminates the targets from backgrounds. To design a proper classification algorithm for this task, accuracy and processing speed are both important. Linear SVM [Vapnik et al. 1998, Fan et al. 2008] is popular for its good generalization ability and its low computational cost. Linear SVM is a supervised classification method that constructs a linear hyperplane to separate data points into two classes. In this stage, we still take the modified HOG as we have used in the previous stage as our feature descriptor. The only difference lies that the anomaly evaluation is unsupervised, while the linear SVM is supervised. By this way, we finally obtain the ship detection result. 3. Relationship with RX Algorithm [Reed et al. 1990] It can be noticed that in GMM if we take the component number K = 1, it will collapse into a single Gaussian distribution model and our anomaly evaluation (3) will simply degenerate to the square of the Mahalanobis distance metric D M (x) between the input patch x and the Gaussian clutter 6

7 Fig. 2: Snythetic data and (a) the proposed mixture model with components K = 2, (b) single Gaussian model. center µ A(x) = 1 N (µ, Σ ) 1 1 =1 exp( (x µ )T Σ 1 (x µ )) ND 2 T 1 (x µ ) Σ (x µ ) = DM (x) (6) where ND is a normalization factor and means the two functions have the same monotony. We also notice that DM (x) have the same form as the classical Reed-Xiaoli (RX) algorithm [Reed et al. 1990], which has been extensively used for hyperspectral image anomaly detection. RX algorithm deﬁnes the anomaly of a spectral vector x as follows RX(x) = (x µ )T Σ 1 (x µ ). (7) Therefore, the Mahalanobis distance metric DM (x) and RX algorithm can be seen as two special cases of our model. Fig. 2 shows the diﬀerence between the proposed method with components K = 2 (a) and single Gaussian model (b) on a group of toy data, where the former one can better catch the nonlinearity and better positioning outliers. 7

8 4. Experimental Results and Analysis We use a set of Landsat-8 satellite images to demonstrate the efficiency of our method. In our experiments, we focus on detecting off-shore ships. Before ship detection, we remove the land regions of the input image by using land masks that are generated by GSHHG database [GSHHG Online]. We use the near infrared band, short wavelength infrared band and long wavelength infrared band of Landsat-8 satellite to detect ship targets Experimental Setup Our algorithm is evaluated on 64 image slices of Landsat-8 satellite image with the slice size of pixels, where 45 of them are used for train and the rest are used for test. A detailed information of these images are listed in table 1. We build an image pyramid before detecting ship candidates for each input image to enhance the detection robustness of different scales. Each image is down-scaled at a rate of 1.0, 1.5, 2, and 2.5 while keeping other detection parameters fixed including the patch size and cell size. Some ships which is too small (whose length are smaller than 6 pixels) have been eliminated. Since we only focus on off-shore ships, the ships that is adjacent with the land are also eliminated. The detection and miss-detection of all the above targets are not included in the statistics. There are 115 ship targets labeled in all for experiments. Since the number of individuals are too small to obtained statistically meaningful training results, we have also collected 2000 individual ship samples from Google Earth images as extra training samples. It should be noticed that also the google earth samples Landsat 8 samples have different sources, the former one can still provide the essential shape and structure information which can be well captured by HOG descriptor for training an SVM model. We set the patch size p = 11, the cell size c = 5 and the patch stride s = 1. We set the component number of GMM K = Overall Results Statistics To test the robustness on different degrees of fringe noise interference, a- mong the five infrared bands of Landsat 8 (5th, 6th, 7th, 10th and 11th), the 7th (short wavelength infrared) and 11th (long wavelength infrared) band are manually corrupted with fringe noise of SNR=10dB and SNR=20dB, respectively. Fig. 3 shows some typical detection results of our proposed method. The yellow bounding-boxes refers to final detected targets. To quantitatively 8

Table 1: Detailed information of the experimental images: Landsat-8 Band ID 5 6 7 10 11 Wavelength Resolution 0.845 0.885µm 30m 1.560 1.660µm 30m 2.100 2.300µm 30m 10.30 11.30µm 100m 11.50 12.

9 Table 1: Detailed information of the experimental images: Landsat-8 Band ID Wavelength Resolution µm 30m µm 30m µm 30m µm 100m µm 100m Solar Irradiance 955W/(m2 µm) 242W/(m2 µm) 82.5W/(m2 µm) - Fig. 3: Some typical detection results of Landsat-8 infrared band image slices. (a)-(b): Near Infrared, (c)-(d): Short Wavelength Infrared, (e)-(f): Long Wavelength Infrared, (g)-(h): Short Wavelength Infrared (SNR=20db), (i): Long Wavelength Infrared (SNR=10db). 9

10 evaluate our method, the precision and recall rate of the detection results are counted. Precision and recall are defined as follows precision = N tp /(N tp + N fp ), recall = N tp /(N tp + N fn ), (8) where N tp represents number of true-positives, N fp represents number of false-positives and N fn represents number of false-negatives. Statistical results are listed in table 2, which suggests an overall high accuracy and stability of our method. Table 2: The precision and recall rate of Landsat-8 detection results Band ID Band Type Noise SNR Precision Recall 5 Near Infrared Short Wavelength Infrared Short Wavelength Infrared 10dB Long Wavelength Infrared Long Wavelength Infrared 20dB average Comparison and Parameter Analysis We compare our ship candidate detection method with two classical methods including spectral-residual saliency detection method (SpecRes) [Hou et al. 2007] and adaptive threshold method (AdaThresh) [Bradski. 2000]. These two methods and their variants have been widely used for detection ship candidates in previous literatures. The spectral-residual saliency detection method analyzes the log-spectrum of an input image and construct the corresponding saliency map based on the spectral residual of the spectral domain. The adaptive threshold method applies an adaptive threshold to an input grayscale image I and transforms it to a binary image BW according to the formulate: { 1 if I(x, y) > M(x, y) BW (x, y) = (9) 0 otherwise, where M(x, y) is the average pixel value of a local square region calculated individually for each pixel. The pixels with values of 1 forms the ship 10

11 recall AdaThresh SpecRes Anomaly-1 Anomaly-2 Anomaly number of candidates Fig. 4: Recall rates of five different ship candidate detection methods under different number of candidates. candidate regions. Parameters of these methods have been carefully tuned to obtain their best detection results. Since the number of component K in our model is an important parameter, detection results under different K are also compared. The single component Gaussian model (K = 1) can be seen as a baseline. Finally, five methods are compared with each other including SpecRes [Hou et al. 2007], AdaThresh [Bradski. 2000], Anomaly-1, Anomaly-2 and Anomaly-3, where 1, 2, 3 represent the component number. Fig. 4 shows recall rates of the above methods under different candidate target number. Clearly, a higher recall rate under a certain candidate number means a better performance. Our radiation anomaly based methods outperform than other two classical methods. Anomaly-3 outperforms than its single-gaussian baseline method Anomaly-1 and is slightly better than Anomaly-2. Considering the computational cost, we do not try more complex model with more than 3 components. We also compare the performance of the three different feature design strategies: HOG+RAW, HOG-only and RAW-only. For a fair comparison, we use the same parameters of feature extraction and the same linear SVM 11

12 precision HOG-only RAW-only HOG+RAW recall Fig. 5: Precision-recall curves of three different features: HOG-only, RAWonly and HOG-RAW. penalty coefficient C=1. We randomly choose half samples for training and the rest for test. Fig. 5 shows the precision-recall curves of the three types of features. We can see a clear advantage of the feature combination. Since the raw pixel value reflects the radiation characteristic of the ground elements while shape of a ship target can be well described by the gradients or edge directions, the raw pixel and HOG combination provide complementary information with each other. This experiment has verified the rationality of our choice. 5. Conclusion We propose a new ship detection method for spaceborne infrared image which is inspired by the biological attention mechanism. The proposed method is designed based on modeling radiation anomalies of an infrared image by converting it into a probability density estimation and outlier detection problem. GMM is used as a probability density estimation model and the EM algorithm is used for parameter estimation. Experimental results for detecting ship targets in Landsat-8 infrared band images are presented and 12

13 analyzed. Compared with other classical ship detection methods, the proposed method suggests higher robustness and higher detection recall rate under the same number of detection candidates. References [Xia et al. 2011] Xia, Y., Wan, S. H., Yue, L. H., A novel algorithm for ship detection based on dynamic fusion model of multi-feature and support vector machine. in Proc. ICIG, [Chen et al. 2011] Chen, F., Yu, W., Liu, X., Wang, K., Gong, L., Lv, W., Graph-based ship extraction scheme for optical satellite image. in Proc. IEEE IGARSS, [Jubelin et al. 2014] Jubelin, G., Khenchaf, A., Multiscale algorithm for ship detection in mid, high and very high resolution optical imagery. in Proc. IEEE IGARSS, [Bi et al. 2012] Bi, F. K., Zhu, B. C., Gao,L. N., Bian, M. M., A visual search inspired computational model for ship detection in optical satellite images. IEEE Geosci. Remote Sens. Lett., 9(4), [Qi et al. 2015] Qi, S., Ma, J., Lin, J., Li, Y., Tian, J., Unsupervised ship detection based on saliency and S-HOG descriptor from optical satellite images. IEEE Geosci. Remote Sens. Lett., 12(7), [Zhu et al. 2010] Zhu, C., Zhou, H., Wang, R., Guo, J., A novel hierarchical method of ship detection from spaceborne optical image based on shape and texture features. IEEE Trans. Geosci. Remote Sens., 48(9), [Ding et al. 2012] Ding, Z., Yu, Y., Wang, B., Zhang, L., An approach for visual attention based on biquaternion and its application for ship detection in multispectral imagery Neurocomputing, 76(1) [Tang et al. 2015] Tang, J., Deng, C., Huang, G.-B., Zhao, B., Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans. Geosci. Remote Sens., 53(3), 1174C

14 [Yu et al. 2014] Shi, Z., Yu, X., Jiang, Z., Li, B., Ship detection in high-resolution optical imagery based on anomaly detector and local shape feature, IEEE Trans. Geosci. Remote Sens., 52(8), [Zou et al. 2016] Zou, Z., Shi, Z., Ship Detection in Spaceborne Optical Image with SVD Networks. IEEE Trans. Geosci. Remote Sens., 54(10), [Tsotsos et al. 1995] Tsotsos, J. K., Culhane, S. M., Wai, W. Y. K., Lai, Y. H., Davis, N., Nuflo, F., Modelling Visual Attention via Selective Tuning, Artif. Intell., 78(1-2), 507C545. [Itti et al. 1998] Itti L., Koch C., Niebur E., A Model of Saliency- Based Visual Attention for Rapid Scene Analysis, IEEE Trans. Pattern Anal. Mach. Intell., 20(11), [Dalal et al. 2015] Dalal, N., Triggs, B., Histograms of oriented gradients for human detection in Proc. IEEE CVPR, [Bishop. 2006] Bishop., C. M, Pattern recognition and machine learning. Springer. [Vapnik et al. 1998] Vapnik, V. N., Vapnik, V., Statistical learning theory (Vol. 1). New York: Wiley. [Fan et al. 2008] Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., Lin, C. J., Liblinear: A library for large linear classification, J. Mach. Learn. Res., 9, [Reed et al. 1990] Reed, I. S., Yu, X., Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution, IEEE Trans. Acoust. Speech. Signal. Process., 38(10), [Hou et al. 2007] Hou, X., Zhang, L., Saliency detection: A spectral residual approach. in Proc. IEEE CVPR, 1-8. [Bradski. 2000] Bradski, G., OpenCV Library., Dr. Dobb s Journal of Software Tools. [GSHHG Online] GSHHG - A Global Self-consistent, Hierarchical, Highresolution Geography Database. [Online]. Available: noaa.gov/mgg/shorelines/gshhs.html. 14

2 Proposed Methodology

2 Proposed Methodology 3rd International Conference on Multimedia Technology(ICMT 2013) Object Detection in Image with Complex Background Dong Li, Yali Li, Fei He, Shengjin Wang 1 State Key Laboratory of Intelligent Technology