Feature Detection Performance of a Bathymetric Lidar in Saipan

Transcription

1 Feature Detection Performance of a Bathymetric Lidar in Saipan LTJG Russell Quintero NOAA National Geodetic Survey - JALBTCX, Kiln, MS russell.quintero@noaa.gov Jerry Mills NOAA Office of Coast Survey - Silver Spring MD Abstract Lidar feature detection performance was analyzed in Saipan using multibeam echosounder data as a reference. The water was optically clear, with lidar data recorded in depths of up to 47m. 772 features were examined with sizes ranging from 0.82m to 18.79m, based on the cube root of the volume. Multiple parameters are examined to identify which most strongly correlate to object detection. The primary parameter for predicting detection is found to be the minimum dimension, either the height or the semi-minor axis, of the feature. Features with a minimum dimension of 2 meters or greater had a 95% + chance of detection. Introduction The purpose of this study is to conduct an evaluation of the object detection capabilities of a SHOALS-1000 bathymetric lidar with respect to a Reson 8125 multibeam echosounder (MBES) survey of one specific geographic region. Similar comparisons have been done in the past, in particular Smith (2006) in Long Island Sound. However, the lidar data acquired for Smith s study was limited to an effective reported maximum operating depth of 10-14m due to water turbidity. In addition to limiting the depths that can be measured, turbidity greatly compromises the ability of lidar to detect objects on the sea floor. In contrast, the clear waters off of Saipan allowed for complete lidar coverage of both the MBES survey area and beyond with recorded depths of up to 47m. In 2006, the Naval Oceanographic Office (NAVO) collected 300% coverage with a 3x3m spot spacing in Saipan Harbor's main navigation channel and approach channel using an Optech SHOALS-1000 lidar system. In 2008, NAVO's Fleet Survey Team surveyed part of the same area with a Reson 8125 MBES. The survey area can be divided into two areas of interest, the main navigation channel which runs roughly east-west, and the southwest-northeast trending approach channel at the western end of the survey area. This study uses the processed and cleaned data provided by NAVO to identify features in the multibeam data and then identifies which of these features are detected in the lidar survey. The goal of this study is to help clarify the capabilities of bathymetric lidar in detecting objects on the seafloor in areas of low water turbidity. The 2008 multibeam survey is referenced to a tidal datum based on a water level gauge in Guam. The lidar survey is referenced to the ellipsoid. No comparison of least depths can be made without conversion between the two datums, the data for which was not available at the time of this study.

2 Figure 1: Multibeam survey is highlighted in gray. The lidar survey is colored by depth. Feature distribution can be seen in the inset. Only 19 of the 772 features were in the long east-west channel. The effects of water quality on this kind of comparison cannot be understated. Using assumed ranges of values for water depth, grid density, incidence angle, and other parameters, Guenther et al. (1996) conducted Monte Carlo simulations to estimate the probability of detecting certain sized objects under varying water clarities. Their results indicated that a 2 meter tall cylinder with a diameter of 2.3m in highly turbid water has at best an 85% chance of detection, which drops to zero at a depth of 8.5 meters. In very clear water, the same object has an almost 100% chance of detection in depths between 8.5 and 21 meters Generalizations about lidar object detection capabilities cannot be made without also discussing water depth and clarity. It is important at this point to ask the question, What is a feature? The International Hydrographic Organization (IHO) Standards for Hydrographic Surveys Special Publication 44 (IHO S- 44) defines a feature as any object, whether manmade or not, projecting above the sea floor, which may be a danger for surface navigation. Moreover, S-44 sets minimum standards for surveys conducted for the safety of surface navigation and considers it the responsibility of each national authority to determine the precise characteristics of features to be detected. Although S-44 states that, for an Order 1a survey, systems must be capable of detecting cubic features greater than 2 meters (2 2 2m) in depths up to 40 meters, charting authorities may specify that smaller features need to be detected to ensure safe surface navigation. Section 3.5 Feature Detection of S-44 also states: "It is the responsibility of the hydrographic office/organization that is gathering the data to assess the capability of any proposed system and so satisfy themselves that it is able to detect a sufficiently high proportion of any such features." Methodology The 2006 NAVO lidar survey and the 2008 NAVO multibeam survey were compared using Area-Based Editor (ABE) and a tool called pfm_feature, both developed in-house by NAVO's Jan Depner. ABE is, as the name suggests, a tool for editing large areas of hydrographic data. It creates a PFM file which contains relevant line, time, position, and depth data. ABE also contains tools that allow the viewing of raw lidar waveform data for any point, concurrent with waveforms for the closest neighbors to the point being examined. Pfm_Feature is a tool that searches through the points in a PFM

3 file and compares each point to the surrounding points, in search of any features that stand out from the local area based on criteria specified by the operator. The two surveys were not conducted concurrently. Efforts are ongoing in Saipan to remove coral heads that pose a particular hazard to navigation. One of the underlying assumptions in this comparison is that while some features may have been removed between 2006 and 2008, no new features were created in that time period. Under this assumption, the multibeam survey used as the baseline for feature identification must be the more recent of the two surveys conducted. Data from both surveys were processed and cleaned by NAVO personnel. The data were used as provided with no additional cleaning to create separate PFM files. The pfm_feature tool was run on the multibeam PFM to generate a list of 947 features, using pfm_feature's IHO Order 1 (2x2x2m) settings. Every feature was then examined, verified, and measured without referencing the lidar data. In the end, 772 multibeam features were retained for this study. Figure 2 shows the distribution of the size of the feature s 3D Characteristic Length, defined as the cube-root of the volume. The average feature size was 2.88 meters, with a standard deviation of 1.96 meters. Figure 2: Distribution of multibeam features as a function of the cube root of the volume. In cases where one, usually large, object was flagged as more than one feature, only the shoalest was retained. Features were also rejected if they were on fliers or noise, if the channel boundary was selected (see Figure 3), or if the object was too heavily shadowed to estimate its extents. Measurements

4 were made of the object s multibeam least depth, the surrounding water depth, and the semi-major and semi-minor axes. Surrounding water depth was defined as the average depth where the seafloor returned to horizontal. When a feature was not sharply defined, the edges were defined to be where the feature became difficult to visually distinguish from the surrounding seafloor. While the edge in Figure 3 is clearly defined, the dark blue band is representative of where the feature edge starts to merge with the seafloor. From these four attributes, additional derived characteristics were calculated in an effort to identify potential detection thresholds (e.g. is there a minimum distance a feature must extend above the seafloor for detection? Is detection more closely correlated to area or height? Is the 2D characteristic length, defined as the square root of the rectangular area, a legitimate parameter for comparing lidar features?) For the purposes of this paper, the height of a feature is the difference between the multibeam least depth and the surrounding water depth, length is the semi-major axis, and width is the semi-minor axis. After all the points were verified and measured, the lidar dataset was opened with the multibeam feature file to search for the identified multibeam features. A positive detection of an object with the lidar was defined as at least one sounding with supporting waveform data from adjacent lines (Figure 4). At this point, all of the data were imported into Matlab for analysis. Two matrices were made, one containing all of the multibeam features and one containing only those detected by lidar. For analysis, features were grouped into 0.25 meter bins, centered such that, for example, the 2 meter bin extended from 1.875m-2.125m. Each bin count in the lidar detection set was then divided by the corresponding bin from the multibeam set to determine the probability of lidar detection based on various parameters to try and identify which parameters, if any, had strong threshold values for detection. Results Numerous scatter plots were created showing the lidar and multibeam features as functions of every pair of parameters to identify any trends in detection. Three parameters showed thresholds below which detection probability decreased sharply: feature height, width, and 2D characteristic length (the square root of the length * width). The 2D characteristic length largely mirrored the results of the width, just with larger values, which is to be expected based on how it is derived. A scatter plot of the height vs. the width revealed an additional trend. As can be seen in Figure 5, the plot of height vs. width can be split into four quadrants. When both are below 2 meters, detection is unlikely. When one

5 of the two is greater than 2m and the other is less than 2m, than undetected features are randomly distributed in axis of the parameter greater than 2m, and when parameters are greater than 2m, detection rate is nearly 100%. Figure 3: The lidar detection threshold can be seen to be controlled by the minimum of the width and height in the plot above. Figure 5 makes it clear that the minimum of the height and the width, hereafter referred to as the minimum dimension, is strongly correlated to lidar detection. Grouping the data into 0.25m bins, and dividing the lidar feature counts by the multibeam feature counts, produces a plot of the lidar detection probability (Figure 6). Detection rates increase steadily as the minimum dimension increased, and cross into the 95% detection ratio at 2 meters.

6 Having determined that the minimum dimension closely correlates with detection rate, the next question was to attempt to determine what other factors might be influencing the detection chance in the low detection region (<2m). More scatter plots were created of every parameter vs. minimum dimension, where the minimum dimension was 0-2 meters. In every case, the undetected features are evenly distributed in the tested parameter (Y-axis), with no clear correlation trend with detection chance. In particular, note the plot of Depth vs. Minimum Dimension (Figure 7). Undetected features are randomly dispersed over the survey depth range of 11 to 15.5, regardless of the feature s minimum dimension, suggesting there is no correlation between water depth and detectability. This indicates that in the depth range of the survey area, with the very low turbidity at the time of the survey, the optical depth was a nonlimiting factor for lidar detection. Additional support for this conclusion is that, within the feature set present in Saipan, there was no maximum dimension (the greater of the height or length for a given feature) that could guarantee detection. As can be seen in Figure 8, there is an area of low detection rate when both the maximum and minimum dimensions are under 2m, and detection rate approaches 100% when the minimum dimension rises above 2 meters. Within the intermediate region, where the minimum dimension is less than 2 meters and the maximum dimension is greater than 2m, the undetected features are randomly distributed, suggesting that a large maximum dimension does not increase the chance of detection if the minimum dimension is too small. Conclusions and Future Work Within the optically clear waters of Saipan, the SHOALS-1000 detected 95%+ of all features with a minimum dimension greater than or equal to 2 meters. Below the 2 meter threshold, detection dropped off linearly as a function of minimum dimension. In addition, water depth, by itself, is not correlated to object detection in very low water turbidity, as evidenced by this study area in Saipan. The minimum dimension of feature height and width appears to be a good parameter to ascertain the lidar detection threshold region and should be included as a metric for evaluating lidar performance in future studies. This appears to address the previously unanswered question of which

7 parameter is more important for lidar object detection, height or areal extent (length * width). Since the publication of the Fourth Edition of the IHO S-44 Standards for Hydrographic Surveys in April 1998 when object detection was first introduced, numerous organizations have attempted to validate the object detection capability of various systems (both sonar and lidar). Tests were created to determine if such systems could detect the specified minimum standard of cubic objects of 1x1x1 meter (Special Order) or 2x2x2 meters (Order 1). To that end, man-made target cubes of various construction materials were created to validate a system. However, such cubes do not appear in nature and as such do not represent the type of objects that hydrographic offices are attempting to identify as potential hazards to navigation. Moreover, in many cases the objects of interest are smaller than the S-44 minimum standards. To quote again from IHO Publication S-44, It is the responsibility of the hydrographic office/organization that is gathering the data to assess the capability of any proposed system and so satisfy themselves that it is able to detect a sufficiently high proportion of any such features." In this study, we have demonstrated an empirical approach to assessing lidar object detection that makes use of new software tools and methods. Recommendations for future work include extending these methods to different areas, different water conditions, and (if possible) different lidar systems. Work should be done to further explore how reducing the spot spacing and increasing the number of flight lines affects object detection performance. Due to a difference in vertical reference, no comparison of least depths could be performed. Future work should include conducting concurrent surveys with the same vertical reference to analyze least depth performance of the lidar system. The results of these studies could then be used to create more robust models than were previously feasible for predicting object detection performance under a variety of conditions. Put differently, better tools and methods for analysis enable better empirical data, which, in turn, enable better models. This will allow hydrographers to better determine the suitability of bathymetric lidar for the survey requirements of a particular area. Acknowledgements: We wish to acknowledge the following individuals who made significant contributions to this work: Jan Depner, Bill Ellenbaas, Jennifer Wozencraft and the staff at JALBTCX; Chris Parrish; Jack Riley; and Michael Gonsalves. References Guenther, G.C., T.J. Eisler, J.L. Riley, and S.W. Perez, Obstruction Detection and Data Decimation for Airborne Laser Hydrography, Proc. Canadian Hydrographic Conference, June 3-5, Halifax, N.S., International Hydrographic Organization. Special Publication No. 44: IHO Standards for Hydrographic Surveys. 5th Edition, Monaco. Smith, S. Empirical Object Detection Performance of LIDAR and Multibeam Sonar Systems in Long Island Sound, International Hydrographic Review, Vol. 7 No. 2, July National Ocean Service (NOS), Hydrographic Surveys Specifications and Deliverables, National Oceanic and Atmospheric Administration, Silver Spring, Maryland.