ESTIMATION OF DETECTION/CLASSIFICATION PERFORMANCE USING INTERFEROMETRIC SONAR COHERENCE

Transcription

1 ESTIMATION OF DETECTION/CLASSIFICATION PERFORMANCE USING INTERFEROMETRIC SONAR COHERENCE Øivind Midtgaard a, Torstein O. Sæbø a and Roy E. Hansen a a Norwegian Defence Research Establishment (FFI), P. O. Box 25, NO-2027 Kjeller, Norway. Oivind.Midtgaard@ffi.no, Fax: Abstract: This paper proposes a new performance metric for interferometric side-looking sonar. With this metric the probability of target detection and classification is reported as a function of coherence instead of across-track range, which is traditionally used. The coherence values are estimated from interferometric sonar processing, and can be converted into equivalent signal to noise ratios. The metric ensures that system performance is estimated based on sensor data quality of the actual mission, and as local values instead of large area averages. The method is demonstrated on synthetic aperture sonar data from HISAS 1030, showing increased consistency between shallow and deep water results, compared to the traditional metric. Keywords: Target detection/classification, Performance estimation, Minehunting, Interferometry, Sonar coherence.

2 1. INTRODUCTION Reliable performance estimates are required for consistent use of the results from target detection/classification systems, as well as for operation planning. For underwater minehunting it is necessary to estimate the total effectiveness over the mission area, based on all individual sonar views of the seafloor. Minehunting operations are to an increasing extent performed with Autonomous Underwater Vehicles (AUV) equipped with high-resolution side-looking sonar, either traditional Side-Scan Sonar (SSS) or Synthetic Aperture Sonar (SAS) [1]. The performance of side-looking sonar systems is typically reported as the probability for correct target detection and classification as function of range, called P(y) curves [2]. These curves depend on the combined properties of the target, sensor, data analysis system (manual or automated), bathymetry, seafloor characteristics and sonar environment. The detection and classification capacity of minehunting systems has traditionally been estimated from a large statistical sample gathered through systematic and comprehensive sea tests with various deployed targets in different environments. Minehunting is usually performed in harbours and littoral waters, where sonar conditions are highly variable. System performance established for other environments may then be inaccurate. Multiple P(y) curves can be prepared to handle expected variations in e.g. seafloor roughness and clutter density. However, preparing curves for every possible combination of the environmental parameters (bathymetry, wind speed, sound velocity, seafloor reflectivity, etc) that affect the sonar performance is impracticable. Also, the mission environment may be complex and rapidly changing in space and time, and perhaps insufficiently known. It may thus be difficult to select the appropriate P(y) curve for a given underwater location. In [2] P(y) curves for a mission were estimated by injecting synthetic target responses into the real sonar images and calculating the performance of Automatic Target Recognition (ATR) algorithms on these ground truth data. The target responses were simulated based on results from a sonar performance prediction tool. Although this approach represents significant progress as it uses the actual mission data to estimate performance, it is still assumed that the local sonar environment can be accurately specified and that the mission area can be divided into a few distinct regions, each with a representative P(y) curve. We propose a new performance metric: probability of correct target detection/classification as function of sonar coherence. The coherence is obtained from interferometric processing of the data from two sonar receivers mounted with a vertical displacement. The coherence can be converted into an equivalent signal to noise ratio (SNR), making it well suited as a system performance parameter. The SNR is commonly used as a sensor performance parameter in simulations [3][4]. The correspondence between coherence and detection/classification probability, P(SNR), can be established through the same approaches as for P(y) curves. Multiple P(SNR) curves may be produced to handle different seabed characteristics. This metric ensures that the performance is estimated based on sensor data quality of the actual mission, and as local values instead of large area averages. 2. SONAR COHERENCE

3 The coherence, γ, is defined as the magnitude value of the zero-lag normalised cross correlation [5] E ss 1 2, 0 1 (1) E s E s where s 1 and s 2 are two co-registered, zero-mean Gaussian random sequences (This is not to be confused with the spectral coherence function). Assuming that s 1 and s 2 are delayed versions of a Gaussian random sequence in additive uncorrelated Gaussian noise s () t s() t n () t 1 1 s () t s( t ) n () t 2 2 (2) the signal to noise ratio, SNR, can be derived from the coherence [5][6][7] SNR 1 (3) The spatial coherence can be calculated from two displaced sonar receivers observing the same seafloor scene. Two receivers with a vertical displacement (baseline) can form an interferometer such as the HISAS 1030 interferometric SAS [8]. In this case, two types of spatial coherence can be calculated: 1) SSS coherence; 2) SAS coherence; There are fundamental differences between the two, in particular regarding along-track resolution. In addition, the temporal coherence can be calculated from ping to ping overlapping elements in the phased array receiver, used in sonar micro-navigation in SAS processing [9]. When baseline decorrelation is compensated for [5][7], the SNR (3) can be used as sonar image quality measure. In shallow waters, the received signals can be contaminated by unwanted multipath where the sonar signals are reflected by the sea surface. Multiple propagation paths to two spatially displaced receivers cause decorrelation or loss of coherence. Similarly, the coherence can only be high for sonar signals without multipath. Hence, the SNR (3) can be used to map the signal to multipath ratio [10]. In areas where the sonar performance is not limited by multipath contamination, the SNR (3) will map signal to additive noise (either self-noise, ambient noise or interference). Excessive noise level or loss of signal (e.g. in acoustic shadow zones) yields low SNR. In this work, we chose the SSS interferometric coherence as sensor quality measure. From observations in shallow waters, this coherence has proven to be a reliable multipath tracer [10]. This measure is also available in the standard processing of the HISAS 1030 SAS. A potential better choice for ATR performance would be SAS interferometric coherence, since this is estimated in the same coordinate system as the SAS image and incorporates possible image artifacts (defocus) induced during SAS processing. This measure will increase the processing time, though. 3. DATA PROCESSING

4 The sonar raw data was processed by FFI s FOCUS toolbox [8] to produce default SSS bathymetry and SAS imagery. The target detector was based on the deformable match filter [11]. It convolves the magnitude normalised SAS image with a generic target signature consisting of a highlight mask followed by a shadow mask. The output is a weighted sum of the two mask responses. Adaptive threshold values are used to segment the match filtered image and connected segmented pixels are clustered into a single detection. Clusters with too few pixels are discarded. The classifier was a Support Vector Machine (SVM) using only two features as input. The first feature was the segment size in the match filter image and the second was the highlight/shadow contrast relative local background variations in the normalised image. The classifier outputs a confidence value between 0 and 1 indicating the detected object s degree of mine-likeness. Targets with confidence value above 0.5 were accepted as correct classifications. This simple classifier was originally developed as a discriminator stage between detection and classification, but was selected for this study as focus is on coherence for performance estimation rather than classifier development. The positions of the target responses were mapped from SAS to SSS data coordinates and coherence values were extracted using a 3 m x 3 m median window for smoothing. The window thus covered both the target and some of the adjacent seafloor. 4. EXPERIMENTAL RESULTS The proposed performance metric is demonstrated on data from the HISAS 1030 sonar mounted on the Royal Norwegian Navy s HUGIN 1000-MR AUV [1]. The data was obtained during six individual missions at two different test ranges in Norwegian fjords. Three of the missions were performed with one MP80 exercise mine placed on the seafloor in deep water (73 and 197 m depth), while the other three missions were performed in shallow water (10-25 m depth) with one MP80 exercise mine and four mine-like cylinder objects ( m length) as targets (Table 1). The surrounding seafloor of all the targets was fairly smooth, but the reflectivity was higher in shallow water due to seafloor pebbles. Mission no No of targets Total no of target views Target depth [m] Sensor altitude [m] Bottom type Mud 2, 3, , 77, Pebbly 5, , Mud Table 1: HISAS data sets used in this study. Figs. 1 and 2 show 10 m x 10 m cropped SAS images of the MP80 target at various slant ranges for deep and shallow water, respectively. The image resolution is approximately 4 cm x 4 cm. Dynamics of the grey-tone scale is set to 45 db for all images. All six displayed target responses were detected and classified as mine-like.

5 Fig.1: On-deck photo of MP80 target and 10 m x 10 m sonar mug shots in deep water (73 m depth). Target slant range is from left to right: 62 m, 128 m and 179 m. Fig.2: 10 m x 10 m sonar mug shots of MP80 target in shallow water (24 m depth). Target slant range is from left to right: 31 m, 76 m and 121 m. In deep water, the target highlight and shadow are easily distinguished from the background seafloor at all ranges, although the shadow contrast is somewhat reduced at far range (179 m). In shallow water, however, the shadow contrast is reduced already at 76 m slant range and is almost indistinguishable at 121 m range. This is due to multipath signals gradually decreasing the SNR with increasing range. Initially, only shadows are degraded, but eventually also highlight contrast is reduced due to the increased background signal level. Fig. 3 shows an interferometric SSS coherence image for a 100 m x 200 m seafloor area in deep water. The image is projected onto ground range with an image resolution of approximately 0.5 m x 0.5 m. The MP80 target at 62 m slant range in Fig. 1 is visible as a small region of lower coherence at along track distance 75 m. Also visible is a larger region of low coherence near the lower image edge. This is caused by an abrupt hollow in the seafloor, approximately 2 m deep, introducing an acoustic shadow along the leading hollow edge. The narrow, horizontal lines with low coherence values are due to the vehicle s acoustical links interfering with the sonar. This illustrates how the coherence can be used as measure of sonar quality, as low values indicate inferior SNR. Overall, the coherence is highest between 40 m and 130 m range, and then slowly decreases towards maximum range. Fig. 4 shows a corresponding interferometric SSS coherence image for shallow water, with the target at 31 m slant range from Fig. 2 visible at approximately 50 m along track distance. The water column is shorter than in Fig. 3, as the sensor altitude was reduced in shallow water (Table 1). The coherence is highest from 15 m to 60 m range, with a maximum value similar to that of Fig 3. However, coherence then drops drastically towards 100 m range and from there slowly decays towards maximum range. This is consistent with the rapid degradation of shadow contrast in Fig. 2.

6 Fig.3: Deep water interferometric SSS coherence. Fig.4: Shallow water interferometric SSS coherence. Figs. 5 and 6 present the single-view results from automatic detection and classification of the targets. Undetected targets were assigned zero classification confidence. The target classification confidences in Fig. 5 are high at short ranges for both deep and shallow water, but decrease at long range. However, the decrease starts earlier and is much more severe in shallow water. From m range there is a large discrepancy between the two P(y) curves. This result corresponds with the coherence images in Figs. 3 and 4. Plotting the target classification confidences and probabilities as function of SNR (Fig. 6) instead of range, yields a better correspondence between the performance curves in deep and shallow water. In both cases, confidence values decrease with decreasing SNR and the response distributions overlap for similar SNRs. This suggests that a large data base of P(SNR) from both deep and shallow waters can be used to estimate performance at any depth, while using a similar P(y) data base would yield estimation errors for both deep and shallow depths due to averaging. The two deep water performance curves in the right plots of Figs. 5 and 6, exhibit a distinct knee at far range and low SNR, respectively. This can be attributed statistical variations due the small number of observations in these parameter intervals. As evident from

7 the left scatter plots, only two deep water target responses failed to be correctly classified. These responses were contaminated by interference and highlight artefacts. A question can be raised whether the poorer classification performance in shallow water could be caused by the rougher seafloor, i.e. pebbles vs. mud. Indeed the P(y) curve for shallow water (Fig. 5) is slightly lower even at short range, but the difference is small compared to the large differences at long range. We thus conclude that multipath is the dominant parameter in our experiment. The proposed performance metric, P(SNR), can also be used for interferometric SSS. It is particularly well suited as an alternative to P(y) for SAS, though, as theoretical along-track resolution in SAS images is independent of range. The fundamental parameter bounding the performance for a given SAS system is thus SNR rather than range. Fig.5: Scatter plot of target classification confidence as function of slant range (left) and P(y) curves for indicated confidence threshold (right). Fig.6: Scatter plot of target classification confidence as function of SNR (left) and P(SNR) curves for indicated confidence threshold (right). 5. CONCLUSIONS We have proposed a new performance metric for interferometric side-looking sonar: the probability of correct target detection and classification as function of sonar coherence. The sonar coherence provides an in-situ measurement of the signal to noise ratio, which is a fundamental parameter bounding achievable sensor performance.

8 This metric has been shown to give more consistent results for a combination of deep and shallow water HISAS 1030 sonar data than the traditional P(y) curve. This preliminary study will be followed by further investigations on the relation between coherence and detection/classification performance regarding estimate robustness and whether SSS or SAS coherence should be used. We will also evaluate methods for incorporating varying seafloor characteristics into the performance estimates. 6. ACKNOWLEDGEMENTS The authors wish to thank the Royal Norwegian Navy for kind permission to use HISAS 1030 data from naval operations with HUGIN 1000-MR AUV. REFERENCES [1] T. G. Fossum, P. E. Hagen and R. E. Hansen, HISAS 1030: The next generation minehunting sonar for AUVs, In Proc. UDT Europe Conference, Glasgow, UK, [2] V. Myers, G. Davies, Y. Petillot and S. Reed, Planning and evaluation of AUV missions using data-driven approaches, In Proc. MINWARA Conference, Monterey, CA, USA, [3] R. Manning, Small Object Classification Performance of High-Resolution Imaging Sonars as a function of Image Resolution, In Proc. IEEE/MTS Oceans 2002, pp , Biloxi, USA, October [4] V. Myers and M. Pinto, Bounding the performance of sidescan sonar automatic target recognition algorithms using information theory, IET Radar, Sonar & Navigation, vol.1 (4), pp , August [5] R. Hanssen, Radar interferometry: Data interpretation and error analysis, Dordrecht, The Netherlands: Kluwer Academic Press, 308 pages, [6] H. A. Zebker and J. Villasenor, Decorrelation in interferometric radar echoes, IEEE Trans. Geosci. Remote Sensing., vol. 30 (5), pp , [7] T. O. Sæbø, R. E. Hansen and A. Hanssen, Relative height estimation by crosscorrelating ground-range synthetic aperture sonar images, IEEE J. Oceanic Eng., vol. 32 (4), pp , [8] T. G. Fossum, T. O. Sæbø, B. Langli, H. J. Callow and R. E. Hansen, HISAS 1030 high resolution interferometric synthetic aperture sonar, In Proceedings of the Canadian Hydrographic Conference and National Surveyors Conference 2008, Victoria, BC, Canada, May [9] A. Bellettini, and M. A. Pinto, Theoretical accuracy of synthetic aperture sonar micronavigation using a displaced phase-center antenna, IEEE J. Oceanic Eng., vol 27(4), pp , [10] S. A. Synnes, R. E. Hansen and T. O. Sæbø, Assessment of shallow water performance using interferometric sonar coherence, In Proc. UAM-2009, Nafplion, Greece, June [11] G. J. Dobeck, Fusing sonar images for mine detection and classification, In SPIE Conference on Detection and Remediation Technologies for Mines and Minelike Targets IV, vol. 3710, pp , Orlando, FL, USA, 1999.