Monterey, CA, USA ABSTRACT 1. INTRODUCTION. phone ; fax ; nps.edu/rsc

Transcription

1 Application of image classification techniques to multispectral lidar point cloud data Chad I. Miller* a,b, Judson J. Thomas b, Angela M. Kim b, Jeremy P. Metcalf b, Richard C. Olsen b b SAIC, 1710 SAIC Drive, McLean, VA, USA 22102; b Naval Postgraduate School, 833 Dyer Road, Monterey, CA, USA ABSTRACT Data from Optech Titan are analyzed here for purposes of terrain classification, adding the spectral data component to the lidar point cloud analysis. Nearest-neighbor sorting techniques are used to create the merged point cloud from the three channels. The merged point cloud is analyzed using spectral analysis techniques that allow for the exploitation of color, derived spectral products (pseudo-ndvi), as well as lidar features such as height values, and return number. Standard spectral image classification techniques are used to train a classifier, and analysis was done with a Maximum Likelihood supervised classification. Terrain classification results show an overall accuracy improvement of 10% and a kappa coefficient increase of 0.07 over a raster-based approach. Keywords: lidar, image classification, multispectral lidar, point cloud 1. INTRODUCTION Classic terrain classification with lidar systems has generally been approached primarily from the basis of the topographic character of the data e.g. X, Y, and Z coordinates and their relationships. Somewhat more rarely, such analysis will take into account the intensity of the lidar returns from the monochromatic systems. There has been some work with multi-wavelength systems, such as the two-color bathymetric system from Airborne Hydrography AB, but this remains largely unexplored territory. Teledyne Optech has recently developed a 3-color lidar sensor, the Titan. Data from this system enable a new approach to terrain classification, entering into the domain traditionally held by passive, raster-based spectral imaging. The sensor collects lidar data in three different wavelengths simultaneously. Ensembles of data points are collected in three different bands. There are several approaches to the analysis of such data. It is possible to rasterize the intensity values from each of the Titan channels and combine them to create a multispectral image a 2- dimensional product. This approach gives up some of the fundamental information in the 3D point cloud, however. A different approach is followed here, where the 3D character of the data are preserved in the analysis phase. The technique described here was initially proposed by Thomas [1] to perform terrain classification using the multispectral point cloud rather than the lidar derived multispectral image. The proposed technique is not intended to ignore the spatial component of lidar data, rather determine if and how much the addition of the spectral component to the spatial component improves terrain classification results over a strictly spectral raster-based technique. Maximum Likelihood supervised classification results from a raster-based technique are compared to results from the proposed point cloud technique based on how well they agree with manually classified ground truth. The overall classification accuracy and kappa coefficient from the resulting confusion matrices are used as the comparison metric. *cimiller1@nps.edu; phone ; fax ; nps.edu/rsc Laser Radar Technology and Applications XXI, edited by Monte D. Turner, Gary W. Kamerman, Proc. of SPIE Vol. 9832, 98320X 2016 SPIE CCC code: X/16/$18 doi: / Proc. of SPIE Vol X-1

2 2. DATA 2.1 Sensor The Teledyne Optech Titan Multispectral Lidar System (Figure 1) takes the ability to seamlessly collect topography and bathymetry found in previous dual-wavelength sensors with near-infrared (NIR) and green lasers, and adds a third shortwave infrared (SWIR) laser. The three lasers are not co-aligned. Channel 1 is the SWIR channel operating at a wavelength of 1,550 nanometers (nm) with a 3.5 degree forward tilt. Channel 2 is the NIR channel operating at a wavelength of 1,064 nm with a 0 degree forward tilt. Channel 3 is the green channel operating at a wavelength of 532 nm with a 7 degree forward tilt. Intensities are recorded with a 12 bit dynamic range. um No_. lolu _=1e bite, Figure 1. Teledyne Optech Titan Multispectral Lidar System (from [2]). 2.2 Flight Figure 2 shows the area of the test flight that Optech performed using the Titan system near Toronto, Ontario Canada in October of Three flight lines covered a neighborhood with a mix of residential houses and vegetation. The flight lines extended over Lake Ontario for bathymetry collection. Data were collected at an altitude of 400 meters with a pulse repetition frequency (PRF) of 200 kilohertz (khz) per channel equaling 600 khz for the system resulting in a point density of 12 points per square meter (Table 1). The maximum PRF of the Titan sensor is 300 khz per channel for a total maximum of 900 khz for the system [2]. The data were processed using Optech Lidar Mapping Suite software with default parameters and raw, unaltered intensity values. A total of nine files in LASer (LAS) 1.2 format were provided, one LAS file per channel per flight line. Proc. of SPIE Vol X-2

3 Figure 2. Titan test flight area October Lake Ontario Toronto, Canada. Table 1. Titan test flight parameters. Parameter Value PRF (per channel) 200 KHz Field of view 30º Scan Frequency 40 Hz Altitude 400 m Speed 150 knots Point Spacing (DT/CT) 0.10 m/0.96 m Point Density 12 points/m 2 3. METHODOLOGY The analysis process depended primarily on a set of procedures designed to bring the 3 laser systems together into point collections that were linked together, with each point assigned a nearest-neighbor return from the other two laser systems. 3.1 Pre-Processing Duplicate points (i.e. points with the same X, Y, and Z values) were found within the point cloud files which caused errors with the nearest neighbor processing. The software tool LAStools lasduplicate was used to remove the duplicate points with the unique_xyz flag [3]. A unique point source ID was calculated for each point cloud by appending the set_point_source flag to the lasduplicate tool command specifying the corresponding flight line and channel. Individual channel point clouds were merged per flight line using the LAStools lasmerge tool (Figure 3). Proc. of SPIE Vol X-3

4 Point Source ID { hghtl.ne 1 Channel 1 Flghtline 1 Channel2 Flghtline 1 Channel 3 Flight line 2 Channel 1 Flghtllne 2 Channel 2 Flight line 2 Channel _ Flghtline 3 Channel _ F light line 3 Channel2 F Iightline 3 Channel 3 Figure 3. Titan point cloud with Point Source ID attributed. 3.2 Noise Processing There were problems found because the three Titan lasers are not co-aligned. In particular, there were issues found with points near the edges of the flight lines after merging the channels. Edge points became staggered between channels causing errors in the nearest neighbor calculation. The LAStools lasboundary tool was used to create a flight line boundary shapefile for each channel and an ArcGIS ArcPy script was developed to merge all of the shapefiles together and output a shapefile for each flight line depicting the coincident interior boundary for all three associated channels. The shapefile was then used with the LAStools lasclip tool to classify all edge points. Figure 4 shows the result of manual classification of all high and low noise points, along with the edge points, using Quick Terrain Modeler software. All noise points were removed from further analysis resulting in 39,456,691 total points within the point cloud. Figure 4. Titan point cloud with noise and boundary edge points classified. Proc. of SPIE Vol X-4

5 3.3 Spatial Classification Points were classified as ground and non-ground using the LAStools lasground tool. Height values were calculated by appending the compute_height flag to the lasground tool command. Noise points were removed by appending the keep_classification flag to the lasground tool command. Vegetation and buildings were classified by entering the lasground output into the LAStools lasclassify tool resulting in four classes: Unclassified, Ground, Vegetation, and Buildings (Figure 5). The entire Titan point cloud was manually classified within Quick Terrain Modeler using the following classes: Unclassified, Ground, Vegetation, Buildings, and Water (Figure 6). The manually classified point cloud was used as ground truth due to the lack of actual ground truth of the survey area. Figure 5. Titan point cloud classified using LAStools lasclassify tool. Unclassified Ground Vegetation _ Buildings Water Figure 6. Manually classified Titan point cloud classified using Quick Terrain Modeler software. Proc. of SPIE Vol X-5

6 3.4 Spectral Incorporation Spectral analysis of the Titan point cloud was made possible by finding the complimentary two channel nearest neighbors for each point and attributing the red, green, and blue (RGB) fields with their respective intensities. Channel 1 (1,550 nm) was set as the blue channel, channel 2 (1,064 nm) was set as the red channel, and channel 3 (532 nm) was set as the green channel as shown in Figure 7. The nearest neighbor calculation was implemented within the MATrix LABoratory (MATLAB) [4] environment using the OpenTSTOOL [5] approximate k-nearest neighbor algorithm and lasdata tools were utilized for reading and writing LAS files in MATLAB. The nearest neighbor routine was performed on single and multiple return points separately and merged together. Figure 7. Titan point cloud nearest neighbor RGB attribution. 3.5 Rasterization A lidar derived multispectral image was created from the three Titan channels by averaging the intensities in 0.5 meter 2 pixels using LAStools lasgrid tool with a value of 50 meters for the fill parameter to fill voids in the resulting rasters. The resulting rasters were clipped to the outer flight line boundary extent and the raster bands were integrated into a three band raster using ArcGIS (Figure 8). A raster classification image was created from the manually classified lidar point values by recording the majority of classification values in 0.5 meter 2 pixels using LAStools lasgrid tool (Figure 9). This ground truth raster was used to access the accuracy of the lidar derived multispectral image supervised classification result. Proc. of SPIE Vol X-6

7 Figure 8. Titan derived multispectral image. Red: Channel 2 (1,064 nm). Green: Channel 3 (532 nm). Blue: Channel 1 (1,550 nm). Figure 9. Classification image created by gridding the manually classified Titan point cloud using a 0.5 m 2 pixel area and recording the majority class of points found in each grid. 3.6 ENVI Preparation Spectral analysis of the Titan point cloud was performed within Environment for Visualizing Images (ENVI) software [6]. The problem with applying traditional spectral analysis techniques to multispectral lidar point cloud data within ENVI is that ENVI expects input data to be in traditional imagery formats. To accomplish the appropriate file format conversion, the processed Titan LAS files were converted to American Standard Code for Information Interchange (ASCII) format using the LAStools las2txt tool with the parse xyzirndecauptrgb flag to keep all the attributes. The ASCII files were then read into Interactive Data Language (IDL) variables and reformatted to a modified band sequential (BSQ) interleave (1 x # of points x # of attributes). A random subset of 20% of the points were flagged to be used as training data and vegetation indices were calculated [7] (Table 2). Table 3 is a complete list of the ASCII file attributes showing the supplied LAS 1.2 Data Record Format 1 attributes (1-13) with the User Data attribute (11) calculated as Height and the custom-calculated attributes (14-21) holding much of the spectral information. Finally, a standard ENVI data file and region of interest (ROI) file of the entire Titan point cloud were output for use in the n-dimensional Visualizer tool. Proc. of SPIE Vol X-7

8 Table 2. Vegetation indices calculated. Index Green Normalized Difference Vegetation Index (GNDVI) Green Difference Vegetation Index (GDVI) Green Ratio Vegetation Index (GRVI) Formula GNDVI = (NIR - Green) / (NIR + Green) GDVI = NIR - Green GRVI = NIR / Green Table 3. List of ASCII point cloud attributes. ASCII Field Attribute ASCII Field Attribute 1 X 12 Point Source ID 2 Y 13 GPS Time 3 Z 14 Red 4 Intensity 15 Green 5 Return Number 16 Blue 6 Number of Returns 17 Manual Classification 7 Scan Direction 18 Green Normalized Difference Vegetation Index 8 Edge of Flight Line 19 Green Difference Vegetation Index 9 Classification 20 Green Ratio Vegetation Index 10 Scan Angle Rank 21 Reduction Flag 11 Height 3.7 N-D Visualizer Within ENVI software, the n-dimensional (n-d) Visualizer tool was used to identify endmember spectra and develop ROI training classes from the Titan point cloud for use in the Maximum Likelihood supervised classification. The n-d Visualizer tool allows for the viewing of multi-dimensional scatterplots and drawing of endmember clusters [8]. In the case of the Titan point cloud data, it is possible to view any number and combination of spatial and spectral attributes. For instance, turning on the X, Y, and Z attributes in n-d Visualizer will display the point cloud in typical spatial fashion and allow rotation of the point cloud about the three axes. The X, Y, and Z view was used to help validate endmember determination. By turning on the X attribute and the data reduction flag attribute, it was possible to display points used for training class determination. By turning on the X attribute and the lasclassify classification attribute, the Unclassified, Ground, Vegetation, and Building classes were easily separated. With the Z attribute and the Height attribute turned on, determination of the land/water/bathymetry interface was trivial (Figure 10). With the RGB attributes turned on, it was possible to determine 7 ground sub-classes: Railroad Tracks, Pavement Marking, Grass, Major Streets, Minor Streets, Sidewalks, and Soil and 4 building sub-classes: Roof 1-4. With the addition of the Vegetation and Water classes, 15 total training classes were identified. Proc. of SPIE Vol X-8

9 Figure 10. Two-dimensional scatterplot of the Titan point cloud in the ENVI n-dimensional Visualizer tool using the Z and Height attributes to classify the land/water/bathymetry interface. 3.8 Supervised Classification Maximum Likelihood supervised classification was performed on the Titan derived multispectral raster image (Figure 11) and on the RGB attributes of the entire Titan point cloud (Figure 12) using a probability threshold of 0.5 and the 13 training classes identified using the point cloud within the n-d Visualizer tool. Maximum Likelihood classification is the most common supervised classification method used with remote sensing image data [9] and calculates the probability that a given pixel, or lidar point, belongs to a specific class providing that the probability is above the probability threshold [10]. Training class ROIs used for the raster classification were drawn spatially from the training class areas identified using the point cloud, thus further sub-setting the training class areas used in the raster classification. The 11 sub-classes (7 ground sub-classes and 4 building sub-classes) from the Maximum Likelihood result for both the raster and the point cloud were merged into Ground and Building classes respectively to match the manually classified ground truth classes. Proc. of SPIE Vol X-9

10 Classification Unclassified _ Ground Vegetation _ Buildings Water 4... ' =. Figure 11. Titan derived multispectral raster image Maximum Likelihood supervised classification result. Unclassified Ground Vegetation Buildings Water Figure 12. Titan point cloud Maximum Likelihood supervised classification result. 4. RESULTS The Titan derived multispectral raster image supervised classification result was compared to the manually classified ground truth classification raster image and the Titan point cloud supervised classification result was compared to the manually classified ground truth point cloud using the final merged Unclassified, Ground, Vegetation, Buildings, and Water classes. Table 4 is the confusion matrix comparing the raster result to the raster ground truth and resulted in an overall classification accuracy of 41% and kappa coefficient of Table 5 is the confusion matrix comparing the point cloud result to the point cloud ground truth and resulted in an overall classification accuracy of 51% and kappa coefficient of Proc. of SPIE Vol X-10

11 Table 4. Confusion matrix comparing the Titan derived multispectral raster image supervised classification result to the manually classified ground truth classification raster image. Ground Truth (Percent) Class Unclassified Ground Vegetation Buildings Water Total Unclassified Ground Vegetation Buildings Water Total Table 5. Confusion matrix comparing the Titan point cloud supervised classification result to the manually classified ground truth point cloud. Ground Truth (Percent) Class Unclassified Ground Vegetation Buildings Water Total Unclassified Ground Vegetation Buildings Water Total CONCLUSIONS The point cloud result showed a 10% improvement in overall classification accuracy and a 0.07 increase in the kappa coefficient over the raster result. The kappa coefficient measures the agreement between classification and ground truth pixels, or lidar points. A kappa value of 1 represents perfect agreement while a value of 0 represents no agreement [11]. Therefore, an increase in kappa value shows an improvement in agreement between the point cloud result and the ground truth. Comparison of the class percent totals between the two confusion matrices shows that the raster result performed better with the Ground (+ 7.5%) and Water (+ 4.86%) classes over the point cloud result whereas the point cloud result performed better on the Vegetation (+ 7.35%) and Buildings (+ 4.81%) classes over the raster result. The additional spatial information of the Z dimension provided by the lidar data plays a more important role in discriminating Vegetation and Buildings than the Ground and Water classes. Spectrally, the Vegetation and Water classes proved difficult to discriminate due to the similarity of spectra within those classes owing to the low spectral resolution of the Titan system. Furthermore, the selection of Titan channels is not ideal for discrimination of vegetation. Calculation of vegetation indices using the green Titan channel rather than a commonly used red wavelength to isolate the red edge of vegetation did not aid in sub-classifying the Vegetation class. Lack of actual ground truth of the survey area also complicated the sub-classification of the Vegetation class. Green light is reflected by water and infrared energy is absorbed by water causing spectrally flat, vegetation-like water signatures and classifier confusion between the Vegetation and Water classes. Proc. of SPIE Vol X-11

12 This study presents a novel method to apply traditional spectral analysis techniques to a multispectral lidar point cloud and shows that the technique can improve terrain classification results over a standard raster technique. Future work on radiometric correction of Titan intensities and actual ground truth collection is envisioned which may further improve terrain classification results using the proposed method. REFERENCES [1] Thomas, J., Terrain Classification Using Multi-Wavelength LiDAR Data, (master s thesis, Naval Postgraduate School, 2015). Retrieved from Calhoun [2] Optech Titan Multispectral LiDAR System: High Precision Environmental Mapping, Teledyne Optech, n.d., accessed online at on 8 April [3] Isenburg, M., LAStools efficient tools for LiDAR processing (version , academic), obtained from [4] MATLAB and Statistics Toolbox Release 2015b, The MathWorks, Inc., Natick, Massachusetts, United States. [5] Merkwirth, C., Parlitz, U., Wedekind, I., Engster, D., and Lauterborn, W., OpenTSTOOL User Manual, [6] ENVI version IDL 8.5 (64-bit) classic mode (Harris Geospatial Solutions, Boulder, Colorado). [7] Broadband Greeness, Harris Geospatial Solutions, n.d., accessed online at on 4 April [8] The n-d Visualizer, Harris Geospatial Solutions, n.d., accessed online at on 30 March [9] Richards, J. A., [Remote Sensing Digital Image Analysis: An Introduction], Springer, Berlin, Germany, 194 (2006). [10] Maximum Likelihood, Harris Geospatial Solutions, n.d., accessed online at on 30 March [11] Calculate Confusion Matrices, Harris Geospatial Solutions, n.d., accessed online at on 30 March Proc. of SPIE Vol X-12