Introduction to LiDAR

Transcription

1 Introduction to LiDAR Our goals here are to introduce you to LiDAR data, to show you how to download it for an area of interest, and to better understand the data and uses through some simple manipulations. In our project we ll be calculating building and tree heights for your project area. The state on Minnesota has flown LiDAR for the entire state. Data may be interactively downloaded via the MN DNR MNTopo viewer ( or for larger areas via the MNGEO ftp site (ftp://ftp.lmic.state.mn.us/pub/data/elevation/lidar/). Raster DEMs, hillshades, contours, and the raw data or point cloud data may be downloaded, in ESRI geodatabases or in commonly supported formats. We will download LiDAR data and derived layers and prepare them for inclusion in our Campus GIS. We will first work with the processed layers, and later the raw data, primarily for building heights. We need to merge together several tiles for the complete St. Paul Campus. This is common when working with LiDAR. Our individual study areas are small, so we ve only a few tiles. Open the MnTOPO website, and familiarize yourself with the tools near the upper right. Use the default display and the zoom button in the upper left, and click- hold- drag to pan, to get the approximate Campus area displayed. Use the download data tool (disk drive with arrow icon, in the toolbar row, upper/left), specify a polygon that is centered on your individual project area. Download the 1 m dem, 1m hillshade the raw lidar point cloud data (these will be.las or.laz (a compressed form) You ll see the DEM and hillshade are rasters, and the raw lidar point data are in files with a.laz extension. The codes are tile number, tiled and compressed to a.laz format. You should copy the DEM and hillshade, and the relevant laz/las files from your download directory to a new directory, in this example we ll call it \LASData. Note that you don t need all the.laz/.las point files that MnTOPO downloaded in the tile. You should refer to the image below, and only copy the files you need for your project area. The boundaries correspond to the tile borders, and the labels the.laz/.las file names. Also note, your working directory should NOT be on a network drive, e.g., the class L drive. LAS files are often quite large, and the overhead of copying and then writing back to the drive across a network is often too slow. Best to work on a local drive, that is, the drive on your computer, or a USB drive plugged into one of the ports. 1

2 Display the DEM and hillshade surfaces. These are largely from an automated process that first tries to identify the bare earth, or ground only returns, and then build a DEM and a hillshade. Notice the odd shapes, sometimes like triangles, on the hillshade surface within the building footprints. These are artifacts of lidar data processing. The algorithms have a hard time identifying the ground near buildings perfectly, so there are often triangular shapes on or near buildings. You also see a roughness and occasional small bumps over grass or forest areas, these are also artifacts of the processing. Also notice the DEM rasters are in UTM15 NAD83(1986). You will need to transform them to CORS96 if you wish to avoid the warnings about datum conflicts when displayed with your other project data. You can do all the following steps in ArcGIS, we will be using LASTools, a widely- used, general purpose lidar processing toolbox that is much faster than processing in Arc, and quite flexible. The UMN has a site licence for LASTools, so we ll be using it for this small exercise. You should make two sub- directories within your \LasData directory, one named buildings, and one named tallveg, into which we ll be subsetting the las data. When you re done, your directory should look something like this window below. 2

3 You notice I ve converted copies of the.laz to.las files. You don t need to, but notice the.laz files are much smaller than the.las. Because lidar data are often so large, there is usually a tradeoff between saving space and processing speed. We re working with a small area, so it doesn t much matter, but in many real projects it does. Also note that I only show a subset of the.las files, your set will depend on your study area. You should know a few more things about LiDAR data. Remember, these LiDAR are from pulses of light sent down from a plane, then reflected back from a target. Precise geometry allows operators to determine the x, y, and z coordinates to within a few inches of each object that reflects back a pulse. There can be many returns measured from each pulse, although we re most often interested in the first returns, to measure object heights, and the last returns, to measure ground elevation. The data are processed before they re delivered, and the collecting organization usually applies algorithms to try to detect ground hits (not all last returns are from the ground), object heights (not all first returns are from the tops of objects), and to provide other information about each data point. Most times each return point contains information on the return number (first, last, or in between), the feature classification (2 = ground, 4 = mid- height vegetation, 5 = high vegetation, 6 = building, 9= water, 0 = unclassified), as well as the scan angle, the strength of the return signal, and other information about the point. LASTools, and most other lidar processing tools, allow us to access these data. We ll use the command prompt to run LAStools. The command prompt is found via the Windows Start button (pink arrow, right), then in the 3

4 Accessories folder > Command Prompt (green arrow and circled, at right) This will open a small window that only allows only text entry (see below). The prompt shows the current path string. Want to set the path to your data location. First type in the drive letter, in my case the Y drive, so I type in the drive letter and then hit return to change to that drive: Y: The prompt should now show the drive, in my case: Y:\ I then use the CD (Change Directory) command to change the active directory, typing in: CD downloads\lasdata So the prompt now becomes: Y:\downloads\lasdata I can type DIR (for directory) and then hit return, it should list the directory contents. I can check it against the Windows Explorer view of the directory, show at the bottom of page 2, to verify I m at the correct subdirectory. Now, on to processing. LASTools is a bit old school, you don t have access to a GUI, you specify options through flags on a command line. Each flag in LASTools starts with a dash, e.g., - i means the following text specifies the input file(s). This is a bit cumbersome for single file processing, but it is great for writing small programs to do repetitive tasks. Another key bit of information is that the asterisk * stands for match everything, and helps 4

5 when specifying multiple files. If you ve copied the LASTools programs from the CLD drive, and are working at home or another non- Skok 35 computer, you ll have to type something like the following line, and enter: set PATH=%PATH%;c:\yourdirectory\lastools\bin where c:\yourdirectory is the path to the directory on your computer that contains the lastools directory. If you re working in 35 Skok, ignore this. The first command we ll type is: Las2las i *.laz odir buildings keep_class 6 This uses the las2las tool to: Read all the.laz files as input (- i means input, the *.laz uses the wildcard symbol * to match anything that comes before a.laz extension). NOTE HERE: IF I ve already uncompressed my.laz files to.las files, I HAVE TO SPECIFY - i *.las as the input Output them to the directory named buildings ( - odir buildings), and Only keep the class labeled 6 (- keep_class 6), in this case the points classified as building points You may get a comment saying the use of LasTools is unauthorized, ignore it. We do have a license, but setting it up requires a few more steps, and the changes they might warn about are minor, and don t change our outcome. We can use CD change to that buildings directory after the command runs, and verify the command worked: Notice the files have the same name as the originals (that s why we put them in a clearly labeled sub- directory), but that they re much smaller, the subset building file is 16,337,316 bytes (16.3 Mb), while the full data set was almost 166 Mb; we ve sampled out only the building returns, and the file is less than 10% of the original size. 5

6 We can apply a similar command to specify keeping only the vegetation classes (3, low veg, 4, medium trees, and 5, high trees): Las2las i *.laz odir tallveg keep_class Note the output directory is set to tallveg here. You should check yourself to make sure the command worked, and note the decrease in size. Convert these las points to ESRI shapefiles with the las2shp command. To convert type in: las2shp i buildings\*.las single_points and hit return. Again, you may get a warning that you re unauthorized, and they re adding small errors to the data, you can ignore them here. The command accesses the las2shp command (las2shp), tells it to use all.las files in the buildings directory as input ( i buildings\*.las), and to output single- point shape files (- single_points), not multipoint files, which cluster points in blocks, reducing file size, but encumbering access. Verify the above command worked, first by looking in the buildings directory and checking that shape file(s) appeared, and then by displaying the data in an ArcMap view: You can see the points, here in yellow, more or less match the building locations. The slight mismatch is because of building lean in the aerial photographs. You might remember that orthophotos correct all locations as if they were on the ground, and the tops of buildings aren t. Thus, the tops get displaced, usually outward from the imaging center relative to the camera position. That s why skyscrapers seem to lean in most vertical orthophotos. 6

7 Now we should look at heights. First, we need to create a variable to hold our eventual height, measured from the ground to the top of the object. Open the table for each of your point shapefiles, and add a new field named zheight, double or float, precision 12, scale 1. This may take some time for each tile, you could try to use the AddField command as an ArcMap batch job. You are strongly encouraged to do this and the following operations as batch jobs (see course resources, or ask Andy or Paul for help). This will greatly ease your work. Next, we need to extract heights from the points. It is usually more efficient to do it as a batch job with the Add Geometry Attributes tool, specify meters and square meters for units, and our project coordinate system. Note that this tool will also add the x and y coordinates for each point. When the process is finished, use the identify button to query a few building heights (z values). Remember, the values are in meters. Do these values make sense? Are the buildings over 200 meters (650 ft) tall? Why do you think you re getting these values? The Z value represents the orthometric height of the top of the building. This is the height relative to our standard surface, near sea level, and not relative to the local ground surface. We need to subtract the local ground surface from the building z value to get building height. One way is to query the DEM raster below each point, and add it to the table record for each point. We can then subtract the DEM height from the z values we calculated from the Add Geometry Attributes, and get a building height. The Extract Values to Points tool, found at ArcToolbox- > Spatial Analyst Tools- > Extract- > Extract values to points does just this. Specifying the input and outputs will query each cell below each point, and add a column to the new data layer specified. This may take quite a long time, so be patient, and again, this process can be run as a batch job for multiple files. 7

8 This generates a new column named RASTEVAL that contains the ground elevations near at each point: (ignore the zheight value in the table at left, this is from a test run, in your files it will be zero or null/not assigned) Now you can calculate the building heights for each point. You can do this manually, but it is better to do it as a batch job with the Calculate Field command. Create a new field called something like build_hght for the height above- ground sampled at each point, and then batch job the field calculation. When you are done, you should have a table that looks something like the one to the right: Now, calculate the average height from point samples for one of your buildings. There are several ways to do this, and it would easiest is with the Spatial Join tool. However, it doesn t work much of the time, freezing up, or worse, returning erroneous answers, with no warning that the values are wrong. There is an alternative which appears to work in most cases, that involves multiple steps. First apply the Intersect tool, with the input building polygons, and the processed LiDAR point file. First, we need polygons for a buildings layer, with IDs you ve assigned (typically short integer), and for clarity, a building name. You should already have created this building layer as part of your database. You may have to display both the LiDAR points and an image of the study area with your building, and adjust the building footprint polygon a bit to account for building lean. 8

9 After ensuring you have a proper buildings file, read the Intersect tool s documentation. Start the Intersect tool, and specify your building and processed (heights calculated) LiDAR points. If you specify All for the join attributes, it will join the attributes by spatial location, into a point file, with the building assigned to each point. We can see the attribute table for the point features below. The last three columns are from the buildings layer, with the building FID, ID, and name, and all the columns before, from FID* through RASTERVAL, are from the point file, including the height for each LiDAR point that landed on the building. 9

10 Now we need to aggregate the points for each building. The easiest way is through the Summary Statistics tool. This summarized features, and I can assign case attributes. This will calculate the summary statistics for every different value of the case attribute. If I make the building ID my case attribute, it will create a table with summary statistics for each unique building. The tool is shown below: Here I ve specified my intersected building/lidar point set as the input, an output table, and the field, statistics, and case attribute. I am calculating the mean height for the building. You might argue I want the tallest point for a building, but most roofs don t vary much in height, and I was concerned that a building might have a tower or scaffold on top, and hence get the a large error. For tree canopies, a maximum statistic would probably make more sense. Running this creates an output table: Note there is a mean height for each case of the building ID. I can then simply join this to my building layer by 10

11 the ID column, and copy/save the joined file to get my height for each building. There are some nuances in applying. Multi- leveled buildings may have to be split into the various pieces, and the heights calculated separately to make much sense. If the building footprints are too large, or there are many errors in classification, then there may be a bit of bias downward. Generally, these errors appear to be small in our application, and this measurement is good enough for our use. Perform these processing steps for both the buildings and the canopy data layer, so that each feature has a LiDAR height. As noted earlier, you should probably calculate the maximum statistic in for the canopy data layer, and the mean statistics for the buildings layer, but evaluate and decide for yourself. Turn in a map of showing your subset LiDAR data for your project area, your building polygon, and label each building with your calculated average building heights. Use an image for the background, either one of the high resolution images on the class drive, or one of the WMS layers. Include the usual title, legend, scale, and other standard map elements. Do this same exercise for the trees in your project area. Again, create and turn in a second map of your results, with an image background, the tree polygons, labeled by height. 11