Airborne Laser Scanning Compliance and Quality Assurance Tool: Development of a Standard Software Procedure and Tool to Quality Assure Elevation Data

Transcription

1 COOPERATIVE RESEARCH CENTRE for SPATIAL INFORMATION 2 Project 2.09/2.15 QA4LiDAR version User Manual Airborne Laser Scanning Compliance and Quality Assurance Tool: Development of a Standard Software Procedure and Tool to Quality Assure Elevation Data Software Developers: Think Spatial & CRCSI Page 1 of 78

2 Revision History QA4LiDAR Date Reviewer Description of changes Version /09/2013 Jessica Keysers First Draft /11/2013 Jessica Keysers Document updates /04/2014 Jessica Keysers Software improvements /02/2015 Jessica Keysers Phase 3 development Beta release /06/2015 Jessica Keysers Use case updates /07/2015 Jessica Keysers Bug fixes and new LAS corruption check components /01/2016 Jessica Keysers Significant processing & efficiency improvements, Pacific Islands software update, upgrade to ArcGIS 10.3 (and hence LAS 1.4) and drop support for ArcGIS 10.1, new bathymetry coverage check /03/2016 Jessica Keysers New license model, upgrade to ArcGIS /09/2016 Jessica Keysers Bug fixes and minor improvements including the QA Report saving edits unless the check is re-run. Addition of new tile origin check. Page 2 of 78

3 Table of Contents List of Tables Introduction Software Purpose What QA4LiDAR Does and Does Not Do Licensing Requirements & Recommendations for using QA4LiDAR Installation Procedure System Requirements Installation Using the QA4LiDAR Form Editor Purpose and Importance Forms in the Acquisition Process Create a New Form Open an Existing Form Filling in the forms Fill in a Tender (*.ctf) Form Fill in a Report (*.crf) Form Save and Print a Form Using the QA4LiDAR Software Supported Data Formats Project Data Version Control Dealing with Very Large Projects Open a QA4LiDAR Project File & Help Menus SETUP: Dashboard SETUP: User Settings SETUP: Project Settings Inputs Processing CHECKS: Automated Checks Survey Control Processing Running Automated Checks Page 3 of 78

4 4.10 CHECKS: Extent Check CHECKS: Visual DEM Check Sampling Multiple Users Check DEM Tiles Shortcut Keys OUTPUT: Report Presence & Reading Forms Report Classification Statistics Survey Control Density / Resolution Flight Lines Vertical Visual Checks (DEM) Output Supporting Information Printing the QA Report OUTPUT: Map Common Tasks Approving the survey control design Troubleshooting Future Improvements Glossary Appendices Appendix 1 How the Automated Checks are performed Project scan in Delivery Completeness & Spatial File Corruption File Naming, Shapefile Attributes & Horizontal Coordinate System Comparison of Report Form to Tender Form Classification Statistics Accuracy of Survey Control Point Density & DEM Resolution Flight Line Coverage Absolute & Relative Vertical Accuracy Page 4 of 78

5 9.2 Appendix 2 How the Visual Checks are performed Extent Checks Visual DEM Checks List of Tables Table 1. ArcGIS 10.3 for Desktop hardware requirements Table 2. Data formats supported by QA4LiDAR Table 3. QA4LiDAR Visual Check shortcut keys Table 4. Survey control output shapefile fields added Table 5. Tile Index output shapefile fields added Table 6. Key term search rules (not case sensitive) Table 7. Tile name in file name search rules Table 8. Openness rating scheme Table 9. Flatness rating scheme Table 10. Control density rating scheme Table 11. Control weighted distribution rating scheme Table 12. Overall survey control rating scheme Table 13. Point density type definitions Page 5 of 78

6 1. Introduction 1.1 Software Purpose This compliance and Quality Assurance tool for airborne LiDAR - QA4LiDAR - is an easy to use, independent, standard compliance and quality assurance (QA) checking mechanism for airborne LiDAR elevation data. By bringing together a set of common checks into one software package and largely automating their implementation, QA4LiDAR simplifies the QA process and increases efficiencies for both the contracting agency and LiDAR provider. Contracting agencies can use QA4LiDAR to perform standard independent compliance and QA testing on their airborne LiDAR data, to ensure providers are delivering data that meets the specification before the project is signed off and completed. LiDAR providers can also use this tool and supply the standard report produced by the tool to the contracting agency as part of the project delivery (in addition to their standard processing and QA procedures). QA4LiDAR is the only package that is focused purely on compliance and QA testing and provides a standard set of checks and an associated compliance report for a suite of airborne LiDAR products. The checks provided are the fundamental checks required, as opposed to a fully comprehensive set of checks. The tool is based on the Australian Intergovernmental Committee on Surveying and Mapping (ICSM) LiDAR Acquisition Specifications and Tender Template version 1.0 November 2010 (ICSM Template) and it s equivalent for the Pacific, hence is specific to Australian, and Pacific airborne LiDAR data collection. If a user does not have access to QA4LiDAR but wishes to perform LiDAR validation, they can refer to Chapter 15 (Airborne LiDAR Acquisition and Validation) of the TERN AusCover Good Practice Guidelines What QA4LiDAR Does and Does Not Do QA4LiDAR DOES enable checks for the following aspects of a LiDAR delivery; Delivery Completeness & Spatial File Corruption Spatial file corruption (las, shp, asc, tiff, tif, ecw, ESRI Grid) Delivery completeness of elements requested as per tender form Tile index coordinate origin Delivery completeness for tiles as per tile index (Orthometric las, Ellipsoid las) Delivery completeness for swath LAS as per flight line shapefile Presence of WDP for full wave form LAS LAS files are requested version and point data record format File Naming, Shapefile Attributes & Horizontal Coordinate System File naming for Australia and Pacific for file formats (las, asc, tiff, tif, ecw, ESRI Grid) Shapefile attributes Horizontal coordinate system of spatial files Delivery completeness for tiles as per tile index (DEM, DSM, aerials, intensity) Delivery Completeness of mosaics (DEM, DSM, aerials, intensity) Validates LAS header information Page 6 of 78

7 Tile size as requested Swath LAS PSID validation Comparison of Report Form to Tender Form Report Form certified Required elements reported as delivered Horizontal coordinate system matches Vertical Reference system matches Compliance with on ground environmental conditions at the time of survey Reported absolute vertical accuracy within specification Reported relative vertical accuracy within specification Minimum Bathymetry Coverage % within specification Minimum Bathymetry Soundings within specification Reported maximum scan angle within specification Geoid model matches Classification level matches Classification Statistics Classified AHD LAS point classification Classified ELL LAS point classification statistics match AHD classification statistics Unclassified LAS data swaths only contain class 0 Total point count of all supplied LAS types matches Accuracy of Survey Control Internal survey control distribution Control density meets minimum requirements Control collection methods suitable Point Density & DEM Resolution Pseudo pulse density meets NPS requirements (reports ground point and all point densities) DEM resolution (asc, ESRI Grid) Bathymetry coverage meets minimum tile % and sounding requirements Flight Line Coverage Gaps between flight lines Scan angle recorded in points within acceptable range Absolute & Relative Vertical Accuracy Absolute (Fundamental) vertical accuracy of LAS and DEM Supplemental vertical accuracy of LAS and DEM Relative vertical accuracy of LAS Extent Checks Visually check the data covers the required extent Visually check for internal voids in the DEM Page 7 of 78

8 Visual Checks Visually check the digital elevation model (DEM) for classification, relative vertical accuracy, surface interpolation, systematic and other errors. Tile management for multiple users Other Version control between subsequent deliveries Enables the use of relative paths for data on external hard drives QA4LiDAR does NOT replace existing LiDAR provider QA procedures, or replace contracting and project management requirements. It also does NOT check the following aspects of a LiDAR delivery; Corruption of aspatial data Naming conventions other than Australian, Pacific & NZ NEDF Folder structures Horizontal accuracy Every LAS point attribute Point density at nadir Spatial distribution of points % flight line overlap DEM hydro-flattening Aerial or intensity imagery values Digital Surface Model (DSM) accuracy Contours; interval, topology, continuity, vertices or smoothness Canopy Height Model (CHM) values Foliage Cover Model (FCM) values Waveform LiDAR other than presence of header link and WDP Compressed LAZ format data Photogrammetric LAS datasets Visual Checks of LAS point data Page 8 of 78

9 1.3 Licensing After installing and running the software you will see the registration screen (shown to the right). Please provide your Machine ID to You will then be supplied with a license key which must be entered in the License box to Register the software. If you re-install the software, you will need to re-supply the same license key to the registration dialog. QA4LiDAR leverages ArcGIS proprietary software. To utilise QA4LiDAR users must have at least an ArcGIS Basic license with the 3D Analyst and Spatial Analyst extensions. All ArcGIS components remain under their original ESRI licensing system. QA4LiDAR also leverages the open source spatial library GDAL, and the LAStools las2las, lasinfo and lasvalidate which are open-source and LGPL ( These are packaged within the installation. The Disclaimer of Warranty states that QA4LiDAR is provided "as is" and without warranty of any kind, express, implied or otherwise, including without limitation, any warranty of merchantability or fitness for a particular purpose. In no event shall the author of this software be held liable for data loss, damages, loss of profits or any other kind of loss while using or misusing this software. Page 9 of 78

10 1.4 Requirements & Recommendations for using QA4LiDAR The requirements to use QA4LiDAR are: Users MUST have at least an ArcGIS Basic license with the 3D Analyst and Spatial Analyst extensions. To be processed as one project, every partial data delivery MUST be integrated into the one overall project directory, overwriting superseded data. Full deliveries are stand-alone project directories. The project extent shapefile MUST be a single (can be multi-part) polygon. The tile index MUST accurately represent the data, i.e. for every tile of data, there needs to be a polygon in the tile index and vice versa. Survey Control data should consist of Ground Control and Fundamental Vertical Accuracy Check Points which MUST be supplied to QA4LiDAR as two separate shapefiles (see section 4.9). Users can also supply custom control and Supplemental Check Points if available. The project extent, tile index and control data shapefiles MUST all have the same coordinate system definition (as identified in ArcMap layer properties) for control checks to work. If the NEDF naming conventions are NOT used; o Key term rules listed in Table 6 MUST be used in file naming otherwise QA4LiDAR will be unable to identify data. o File naming of tiled data MUST match one of the tile name search rules in Table 7 for QA4LiDAR to identify tiled datasets. The Output Folder selected; o MUST have sufficient available disk space (DEM mosaic requires ~one twentieth the size of the DEM, other outputs require a relatively small amount of space). o MUST be located outside the project folder. o As it is shared between users, if there will be multiple users of the project, it should be located on a network drive so everyone can access it. If there will only be a single user for the project, the Output Folder should be located on the local machine or USB 3 connected external disk for increased speed. The Working Directory selected; o MUST have at minimum, the size of the classified AHD LAS dataset in available disk space. o MUST be located outside the project folder and separate to the output folder. o As it is user specific, it should be located on the local machine or USB 3 connected external disk for increased speed. The location of project data MUST be appropriate to perform the Visual Checks with greater than one user i.e. it cannot be on an external hard drive that is only accessible to one user. File and folder names should NOT contain spaces (use underscores - this is common practice for spatial data). Project files should NOT be open in other software as access issues may occur. Avoid opening instances of ArcGIS on the computer while QA4LiDAR is running as conflicts may occur. If data FAILs the corruption check in the first check group with an error (as opposed to a warning), it MUST be fixed (or removed and the project rescanned) before running the following checks or it may cause them to crash. Page 10 of 78

11 The recommendations for using QA4LiDAR are: The QA4LiDAR Tender and Report Forms are not required but they are recommended. It is HIGHLY recommended to run the Delivery Completeness & Spatial File Corruption check first to check and fix any corruption or delivery issues. Topographic and bathymetric data should be run as separate projects. If unclassified swath LAS and/or waveform LAS do not exist in your dataset, un-ticking the sub checks within the first two check groups related to these data types is recommended to save time. The recommended naming convention is NEDF. Attributes used in the relevant shapefiles should be the EXACT, case sensitive field names as per the Tender Form for the shapefile attributes check to pass. However, if not this will not adversely affect other checks and can be conditionally passed. If processing data from and/or to external disk, a USB 3 connection (disk and port) is HIGHLY recommended. If attempted over USB 2 or a network connection processing will be extremely slow. Otherwise data should be processed to/from the local machine. The use of long folder path names for project data, the output folder and the working directory should be AVOIDED otherwise files may not be able to be written successfully. Ideally less than 256 characters (this is common practice). Once a QA4LiDAR project is created, the data within the folders should NOT be renamed or rearranged as project links will be broken. The user is however able to rescan the project directory to identify changes (refer to section 4.2). Page 11 of 78

12 2. Installation Procedure 2.1 System Requirements QA4LiDAR is a standalone, application for Windows 7, 8 and 10. It will run on 32 bit and 64 bit machines. It requires ArcGIS and the Microsoft.NET Framework version 4.0 or higher to be installed on the computer. A minimum of 100 MB of disk space is required for installation. When processing data, additional disk space will be required for temporary files and output files (see section 0). Hardware requirements can be considered the same as for ArcGIS (Table 1). Table 1. ArcGIS 10.3 for Desktop hardware requirements CPU Speed Processor Memory/RAM Display Properties Screen Resolution Swap Space Disk Space Video/Graphics Adapter Networking Hardware 2.2 GHz minimum; Hyper-threading (HHT) or Multi-core recommended x86 or x64 with SSE2 extensions 2 GB minimum (for large datasets, the more RAM the better i.e. for a 1TB project, 16GB RAM is recommended) 24-bit colour depth 1024 x 768 recommended minimum at normal size (96 dpi) Determined by the operating system; 500 MB minimum. 2.4 GB. In addition, up to 50 MB of disk space may be needed in the Windows System directory (typically, C:\Windows\System32). You can view the disk space requirement for each of the 10.3 components in the Setup program. 64 MB RAM minimum, 256 MB RAM or higher recommended. NVIDIA, ATI, and Intel chipsets supported. 24-bit capable graphics accelerator OpenGL version 2.0 runtime minimum is required, and Shader Model 3.0 or higher is recommended. Be sure to use the latest available driver. Simple TCP/IP, Network Card, or Microsoft Loopback Adapter is required for the License Manager. 2.2 Installation QA4LiDAR Form Editor 1. The QA4LiDAR Form Editor is a standalone installation. 2. Obtain the installation package from 3. Run the QA4LiDAR Form Editor msi file to install the editor. The QA4LiDAR Form Editor setup wizard will guide you through the installation. 4. New versions of the editor will overwrite existing versions when installed. 5. To uninstall use Control Panel > Programs and Features. 6. Any software updates will be available at QA4LiDAR 1. Obtain the installation package from the CRCSI. 2. Run the QA4LiDAR Setup msi file to install the tool. The QA4LiDAR setup wizard will guide you through the installation (click yes to all the windows that appear). 3. New versions of the tool will not overwrite existing versions when installed. 4. To uninstall use Control Panel > Programs and Features. 5. Any software updates will need to be supplied by the CRCSI. **Please ensure that QA4LiDAR and the QA4LiDAR Form Editor are installed in separate folders (as per the default installation directories). If the installation files are mixed within the same folder, crossover may occur between the Forms and QA Report displays.** Page 12 of 78

13 3. Using the QA4LiDAR Form Editor 3.1 Purpose and Importance QA4LiDAR is intended to complement the ICSM LiDAR Acquisition Specifications and Tender Template. In order to check a LiDAR delivery against the specifications defined in the template, QA4LiDAR requires a condensed form version of the template from which to automatically extract project specifications. This is referred to as the Tender Form (*.ctf extension) and is completed by the contracting agency. To be able to check project results against the specifications and data delivered, QA4LiDAR also requires a similar condensed form version of the final project report from which to automatically extract project results. This is referred to as the Report Form (*.crf extension) and should be completed and certified by the LiDAR provider. The information supplied in these forms MUST match the relevant information in the full tender and project report. QA4LiDAR is able to run without the forms however they are recommended, as without them, some checks will not be possible and no Pass/Fail results will be returned. If you do not have access to the forms but have sufficient information, you can create the forms in hindsight to obtain Pass/Fail results. If the forms are not used with QA4LiDAR, no Pass/Fail results will be output and only the following checks and results ARE possible; Corruption checks on spatial files Tile index coordinate origin All tiles for tiled datasets have been delivered LAS version and PDRF Swath LAS PSID validation Presence of WDP for full wave form LAS Validate LAS header information Classification Statistics checks Survey Control; Density & Distribution Vertical Accuracy; Absolute and Supplemental accuracy of LAS and DEM, Relative vertical accuracy of LAS Flight Lines coverage raster Density; point density statistics and rasters, DEM resolution, bathymetry coverage statistics Visual checks The following checks are NOT possible without the forms; All Pass/Fail results that are based on the specification Delivery Completeness File naming checks Horizontal Coordinate System (HCS) of data matches the Tender Form Shapefile Attributes Tile size Tender Form versus Report Form checks Classification Must Have and Must Not Have point classes Survey Control collection method Scan angle of points Page 13 of 78

14 3.2 Forms in the Acquisition Process The contracting agency fills out the Tender Form along with their customary version of the ICSM LiDAR Acquisition Specifications and Tender Template and supplies these to the LiDAR provider. Along with the LiDAR data delivery, the LiDAR provider completes the traditional project report and the QA4LiDAR Report Form. The LiDAR provider and contracting agency are both able to run QA4LiDAR. QA4LiDAR can also be run without supplying the forms as input. The Form Editor is versioned to match the main software and ensure compatibility between the two. Hence if a new version of QA4LiDAR is released, it may be necessary to regenerate forms with the equivalent version of the Form Editor. 3.3 Create a New Form Run the QA4LiDAR Form Editor (a separate installation to the main QA4LiDAR software). Click on the button for the type of form you wish to create i.e. Tender or Report. Alternatively use the File dropdown menu to choose New then New Tender Form or New Report Form. A new, blank form will appear. 3.4 Open an Existing Form Within the QA4LiDAR Form Editor, from the File menu, select Open. Locate your Tender Form (*.ctf) or Report Form (*.crf) file and select Open. Page 14 of 78

15 3.5 Filling in the forms Most elements on the forms have a Tool Tip, which if hovered over with the mouse, will provide additional information about that element and how to address it on the form. The Home menu provides a Zoom panel to navigate around the form and a Print panel from which to preview and print the form Fill in a Tender (*.ctf) Form Firstly, filling in the project details section at the top of the form is recommended. This includes contract number, project title, the date the contract was issued, who the tenderer is (who is completing the form), the contracting company and their address. Under the headings Datasets and Reports & Ancillary Information, tick the check boxes for the elements you require. You will notice that some elements are ticked by default and cannot be unticked. These elements (Report Form, Ground Control, FVA Check Points and Tile Index) are required to use QA4LiDAR (the Report Form is required if using the Tender Form) and have default values provided. You will also notice some elements are ticked by default but may be un-ticked if not required. These elements (LAS, DEM, Project Report and Metadata) are the recommended base products for every LiDAR delivery. Three of these elements appear in red meaning the form is not valid, unless validated by supplying a data format or un-ticked. *Note. If you are requesting multiple DEMs of different resolutions, the primary DEM should be specified with the tick box and resolution slider (under the Coordinates & Accuracy heading), and any additional DEMs should be specified under the Custom Datasets heading. Only the primary DEM will be used in the checks. For each element you tick, use the Format dropdown box to the right to specify the required format(s) (where applicable). You are able to select more than one format per dataset where required. If you ticked Digital Elevation Model, use the Hydro-Flattening dropdown box below it to specify whether hydro-flattening is required. If you ticked Contours, Ground Control, FVA Check Points, SVA Check Points, Tile Index or Flight Trajectory, use the associated Attributes dropdown box to specify which attributes you require for each dataset. You are able to select more than one attribute per dataset. Field name attributes used in the shapefile should be the exact, case sensitive attributes specified on the Tender Form so QA4LiDAR can locate the fields. They are not editable in an attempt to standardise fields. Page 15 of 78

16 There is a Custom Datasets option, for you to add additional required elements that do not appear in the standard list. Add a Custom dataset and select or type the delivery element and format. You can add multiple custom datasets. Custom elements will not be checked by QA4LiDAR but they help provide a complete project summary for the contractor and provider. Under the heading Other Requirements, use the dropdown boxes and slider bars to specify the details for each element. If there are Environmental Conditions, please specify the page number at which details can be found in the tender contract., The Class Requirements are defined using a grid similar to an aeroplane seating chart. If there is no requirement for a class, leave it white (it will be reported as an Ignored class on the QA4LiDAR QA Report). If you Must Have a class, click once on its number (e.g. 2 - Ground) so it turns green (it will be reported as a Required class on the QA4LiDAR QA Report). If you Must Not Have a class, click twice on its number so it turns red (e.g. 12 Overlap Points which is red by default. It will be reported as an Unwanted class on the QA4LiDAR QA Report). If a class above 12 (i.e. Other as specified ) is selected as Must Have, a definition for this class must be typed in the free text box that appears. Under the heading Coordinates & Accuracy, use the dropdown boxes for Horizontal Coordinates, Vertical Reference, and Geoid Model to specify your requirements. For Absolute Vertical Accuracy, Relative Vertical Accuracy, and DEM Resolution use the slider bars to specify your accuracy (@ 95% confidence interval), and primary DEM resolution requirements. The default values set are typical values for a LiDAR survey. Page 16 of 78

17 When your form is complete and valid (i.e. no red text), Save the CQT and print it to PDF (refer to section 3.6) to supply it to the LiDAR provider. The CQT file can also be supplied to the LiDAR provider if required. Saving the form will create a Last Save date stamp in the top right corner so the user is aware of when the form was last edited. To save time completing a Tender Form, you may open a previously filled in Tender Form from a similar project, edit the values as required, and choose to Save As. If using this method, be careful to correctly edit all necessary values Fill in a Report (*.crf) Form Firstly, filling in the project details section at the top of the form is recommended. This includes contract number, project title, the delivery date, delivery volume unique identifier, who the contractor is (who is completing the form), the company contracted and their address, and the contractor job number. Page 17 of 78

18 Under the headings Datasets and Reports & Ancillary Information, tick the check boxes to confirm the elements you are delivering for the whole project (even if supplying in partial deliveries). For each element you tick, use the Horizontal Coordinates dropdown box to the right to specify the horizontal coordinates used (where applicable). When you specify the horizontal coordinates for one dataset, the rest will be populated with the same value to save time. Edit these pre-populated values if required. For Digital Elevation Model, Ground Control, FVA Check Points and SVA Check Points use the Vertical Reference dropdown box to specify the vertical reference system for those elements. There is a Custom Datasets option for you to add additional elements you are supplying that do not appear in the standard list. Add a Custom dataset and select or type the delivery element and format. You can add multiple custom datasets. Custom elements will NOT be checked by QA4LiDAR but they help provide a complete project summary for the contractor and provider. Under the heading Other Requirements, use the dropdown boxes and slider bar to specify the details for each requirement. Indicate if you complied with the Environmental Conditions in general for the project. If there was a small area affected differently or an issue that needs to be mentioned, use the free text space provided to comment or reference the relevant page in the full project report for further detail. Under the heading Coordinates & Accuracy, use the dropdown boxes for Ground Control Method and FVA and SVA Check Point Method to specify the methods used for data collection for these types of survey control. Use the free text box associated with each to briefly explain how each type of control was connected to the datum or refer to the relevant page in the project report. For Absolute Page 18 of 78

19 Vertical Accuracy LiDAR Point Cloud, Absolute Vertical Accuracy DEM, and Relative Vertical Accuracy LiDAR Point Cloud, use the slider bars to specify the accuracies achieved 95% confidence interval). Use the dropdown box for Geoid Model to select which geoid model was applied to the LiDAR. Indicate whether any corrections additional to the geoid model were applied, the size of the shift if a vertical constant shift was applied, and use the associated free text box to explain the additional corrections or reference the relevant page in the project report. Finally, certify the Report Form as a certificate of delivery. Page 19 of 78

20 When your form is complete and valid (i.e. no red text), Save the CQR and print it to PDF (refer to section 3.6) to supply to the contracting agency. The CQR file can also be supplied to the contracting agency if required. Saving the form will create a Last Save date stamp in the top right corner so the user is aware of when the form was last edited. To save time completing a Report Form, you may open a previously filled in Report Form from a similar project, edit the values as required, and choose to Save As. If using this method, be careful to correctly edit all necessary values. 3.6 Save and Print a Form From the File menu or Home panel within the form you are working on, select Save. To save a new version select Save As then name and choose the location for your new Tender (*.ctf) or Report (*.crf) file and click Save. You will only be able to save if the form is valid i.e. all required areas have been filled out and there is no red text remaining. From the Home panel within the form you are working on, select Print to PDF. A Save As dialog will appear for you to choose the output directory and file name. Once the PDF has been created, you can open it in Adobe to print a hard copy. The PDF lists the sections of the form as bookmarks in the left panel for easy navigation. Page 20 of 78

21 4 Using the QA4LiDAR Software 4.1 Supported Data Formats QA4LiDAR supports the data formats listed in Table 2. It does NOT perform checks on other data formats that may exist within project folders. To save processing time such files are ignored. The reporting of file counts and sizes on the Dashboard only include the file formats listed below. Table 2. Data formats supported by QA4LiDAR. Data Type Supported Format/s Point Cloud LAS (versions ) Unclassified swath LAS waveform + WDP (only for presence)** DEM, DSM ESRI GRID ESRI ASCII Intensity Imagery ECW Tif & GeoTiff ESRI GRID ESRI ASCII Aerial Imagery ECW Tif & GeoTiff Contours, Flight Trajectory, Shapefile (preferred) Tile Index GDB Feature Class ArcInfo Coverage MapInfo TAB Ground Control Shapefile CSV GPS Base-station Data Rinex v2.11 Rinex v3.0 Metadata ANZLIC XML NEDF XML PDF Word (.doc and.docx) Project Report PDF Word (.doc and.docx) Excel (.xls and.xlsx) QA4LiDAR Tender Form CQT QA4LiDAR Report Form CQR Other Any other data formats will not have any checks performed on them * QA4LiDAR uses free LAStools and ArcGIS to handle LAS files. The LAS 1.4 standard states it has "Backward compatibility with LAS 1.1 LAS 1.3 when payloads consist of only legacy content ( pages 2 & 3). ArcGIS 10.2 supports LAS versions 1.0, 1.1, 1.2, and 1.3. Additionally, LAS version 1.4 files that are 1.3 compliant, containing point record formats 0 through 5 are supported. LAS version 1.4 files containing point record formats 6 through 10 are not supported. Hence backwards compatible LAS 1.4 files (only containing point record formats 0 through 5) are supported by QA4LiDAR, but those containing record formats 6 through 10 are not supported. ArcGIS 10.3 and above support LAS version 1.4. **If a project contains waveform LAS of a type other than unclassified swath, this may cause the checks to fail as such data is untested. Page 21 of 78

22 4.2 Project Data Version Control QA4LiDAR REQUIRES all project data and files to be within a single parent directory. The first time a project is scanned into QA4LiDAR, the New Project option is used. If a subsequent (partial) delivery is made for the project it must be placed within the existing project directory and the Open Project and Rescan options used. Partial deliveries could occur for large projects delivered in stages, or for re-supply of a subset of failed data (e.g. LAS files) by the provider. In the case of re-supply, QA4LiDAR requires the user to remove the superseded data from the project directory and replace it with the new data. It is recommended that a delivery number (e.g. 2) be included in the folder name for that partial delivery element as explained in the following paragraphs. Retaining superseded data within the project directory is NOT recommended as may significantly increase the size of the project and slow down the scan and checks. When the user opens an existing project with changed files, they MUST then select to Rescan Project Directory so the changes to the project directory can be identified by QA4LiDAR. QA4LiDAR does NOT record all previous (i.e. an entire history) of scans/rescans of a project directory. It DOES retain record of the scan immediately prior to the current scan. Therefore, it is able to detect changes in the project directory between two consecutive scans i.e. files that have been added or removed from the project. QA4LiDAR version control operates primarily for DEM files for the Visual Checks. The system uses unique identifiers (hashes) based on the binary code within each DEM (and other raster) files as well as DEM file names to manage versions of the DEM files in the project folder between two consecutive scans. For LAS files, the scan/rescan only detects new or removed files based on file creation/modification dates, file size, and file paths and file names. Hence you may want to use a naming convention that includes the delivery number in the folder name, especially for data types other than DEM such as LAS. Using this information, the program will inform the user of the total number of files in the project, the number of new files, and the number of removed files between consecutive scans. From the binary DEM data, QA4LiDAR can detect if DEM data has changed between scans and whether it is necessary to regenerate the DEM mosaic for Visual Checks. If a provider accidentally redelivered the same DEM files, the files will have the same unique identifiers in QA4LiDAR, so a new mosaic will not be generated and files will NOT be reported as new/removed unless the pathname was altered. For other file types, it is more difficult to detect change using only file paths and file names. If for example the project LAS files were updated and re-delivered and replaced the old LAS files in the project folder using the same name and path, they would NOT be detected as a change and would not be reported as new/removed on the Project Summary however checks could be rerun on them and may return different results. If the user wished QA4LiDAR to detect this change and report it in the Project Summary, they could slightly alter the path name of the new LAS files by adding a delivery number (while still removing superseded data). Changes CANNOT be detected between non-consecutive scans. For example, if the provider accidentally re-delivers the original data (scan one) as the second re-delivery (scan three), QA4LiDAR will be unable to identify this as part of the scanning process. However, the error could be detected by the user when automated checks provide the same results as the original delivery. Page 22 of 78

23 If a LiDAR project is collected and delivered in stages, each subsequent delivery should be added to the single parent project directory and the directory re-scanned. However, if it is a very large project it may be more efficient to process the areas separately - refer to section 4.3. Sometimes in staged deliveries, the entire project is re-delivered at completion. To check this final complete delivery is the same as the previously scanned project of all integrated staged deliveries that passed QA, the user can locate the final complete delivery at the exact directory location of the staged delivery project folder (don t delete the integrated staged delivery, just re-name/re-locate it first) and rescan the project folder. If the total number of files in the project matches and there are no new or removed files reported, the final complete delivery can be considered the same. This avoids rerunning the full set of Automated Checks. To perform Visual Checks on re-delivered DEM data the user would want to target only the DEM tiles that had been re-delivered. As mentioned the DEM mosaic will be regenerated if DEM tiles have changed, however the system will not automatically recognise which tiles these were as part of the sample selection. The solution to this is to load the tile index with the first set of Visual Check results and use the failed tiles as the flag for sample selection (refer to section ). 4.3 Dealing with Very Large Projects Very large LiDAR projects are usually collected and delivered in stages. Rather than combining all the data into a single project directory for QA4LiDAR, it is recommended that each stage be processed as an individual project. Large datasets can take a very long time to process and ArcGIS processing tools do not deal well with very large volumes of data. The size that can be handled will depend on a lot of different variables. These include the specifications of your computer (RAM etc.), other applications running at the time, the location of project data (USB, network etc.), dataset properties i.e. products delivered, point density, tile size, DEM resolution etc. Testing has revealed that the software can handle projects of 2,000 x 1km tiles (it may handle larger projects). If there are areas a lot larger than this delivered, they may need to be split into chunks, for example perhaps based on Local Government Areas. In order to process each stage/area as an individual project, the relevant files specific to each stage/area will need to be created i.e. an extent shapefile, tile index, control shapefiles, flight line shapefile, that only have data for that area. The same Tender and Report Forms can be used for each stage/area. Page 23 of 78

24 4.4 Open a QA4LiDAR Project Run the QA4LiDAR software. The splash screen then project screens shown below appear. The user can choose to create a New Project (this means a new QA4LiDAR *.cqp file) or if a project (*.cqp file) already exists, Open Project. The folder pointed to, should be the top level folder containing your LiDAR project data. *Note. It is highly recommended that the project data is located on the local machine or a USB 3 connected external hard drive so that the scan in of data is not occurring over a network or slow connection. If using USB 2, the scan process will run extremely slowly. However, if the project requires multiple users for the Visual Checks it will have to be on a network drive. If the user selects to create a New Project, the project directory will be scanned. This scanning process creates a catalogue of all the files in the project directory which is known as the QA4LiDAR Project Database (*.cqp). The scan determines things such as the data type, whether a file is a tile, what coordinate system a spatial file is from the file name (using NEDF conventions) and so on. If a QA4LiDAR Project Database already exists at the new project location selected, the user can choose to Open the existing project or Overwrite it. If the user chooses to overwrite, the existing QA4LiDAR Project Database (including any results) will be deleted and replaced with the new scan results. Otherwise the user can select to Open Project (an existing project). Page 24 of 78

25 4.5 File & Help Menus The File dropdown menu allows the user to start a New project, Open an existing project, Close the current project, Print Report Summary (the standard QA4LiDAR QA Report), Print QA Errors (detailed information about Failed checks), Save Results as XML, and Exit the software. The Help dropdown menu provides access to the PDF help documents including the Quick Start Guide, the Shortcut Key Guide for Visual checks, this User Manual, the Map Legend and the Video Tutorials on YouTube. It also gives version information About the software. 4.6 SETUP: Dashboard Once a project is open, the below Dashboard Project Summary page will appear. This Dashboard can be accessed under the SETUP heading on the left menu panel. QA4LiDAR is designed to step through the side panel menu in order starting from the Dashboard and progressing down through each subsequent screen as per the following instructions. The Dashboard provides the dataset path and ArcGIS version, results of the last scan giving a breakdown of the number of files for each file type and the file size for each file type, whether there are any active QA sessions, as well as the total number of files in the project and the total file size. If a rescan has been performed, the number of removed files, new files, and changed DEM files are displayed (these numbers only include the supported data formats as described in section 4.1 other formats are ignored). The rescan button is explained below. The QA4LiDAR status including version number, ArcGIS license type and path of the tile index are also provided along with links to the output files created by the software (once they are generated). The user has the option to Rescan Project Directory. A user may choose to rescan the project directory if changes have been made to the directory i.e. if new data has been added or superseded data removed. An example of this would be a large project delivered in stages, or a subset of the project updated and re-supplied by the provider. Rescanning a project identifies changes to the project directory and is quicker than overwriting and starting the QA4LiDAR Project Database from scratch. However, users should be aware that if they rescan the project and go directly to the Report without re-running the checks, any results that appear in the Report may not be valid for the new data until relevant checks are re-run. QA4LiDAR stores relative paths even though it displays full paths. Therefore, it is possible for the user to Open a project for which the drive letter or a top level folder name has changed e.g. because the data is on an external disk. The Project Settings (section 4.8) should still appear correctly with green ticks. However if not the user can rescan the project folder to update the QA4LiDAR Project Database and will also have to resupply the extent, forms and control which will have warning symbols next to them. If there are warning symbols and the user just resupplies these parameters and doesn t rescan the project, the QA4LiDAR Project Database paths will be different to the existing data and the checks will fail. Page 25 of 78 Save Results As XML

26 4.7 SETUP: User Settings Once a project has been opened or created, the User Settings, which are located under the SETUP heading on the left menu panel, MUST be completed. The user s full name is required and supply of an address is recommended. This information is used by the Visual Checks section to handle multiple users. The user information is recorded per QA4LiDAR session within the project database and can be changed during an active session. It is used to track tile sample selection per user and to record which user checked each DEM tile, so that clarification of Visual Check error mark-up is possible if required. It is also used to ensure that only one Automated Checks session is being run at a time. These details are saved to the user profile for the computer/user and will appear by default if they have been filled in previously. Page 26 of 78

27 4.8 SETUP: Project Settings Once the User Settings are complete, the Project Settings, which are located under the SETUP heading on the left menu panel, MUST be completed. Once complete, the project settings can be saved to HTML using the Export button at the top of the screen. If a new project for the same data needs to be started, the user can Import these saved settings to save time filling them in manually again Inputs A Project Extent polygon shapefile is REQUIRED, which should represent the specified capture extent for the project and be in the SAME coordinate system as project data (as per the ArcMap layer properties dialog). It is then optional, but highly recommended to supply the QA4LiDAR Tender Form (*.cqt) and QA4LiDAR Report Form (*.cqr) Processing The Output Folder is where raster and shapefile outputs will be stored. If a check that produces a raster or shapefile etc output is run a subsequent time, the existing raster or shapefile etc for that check will be overwritten. Please ensure that the output location selected is writeable and has enough free space for storage (the DEM mosaic requires about one twentieth the size of the DEM dataset, while other outputs require a relatively small amount of space). The output folder is per project. If there will be multiple users for the project, the output folder needs to be in a location accessible by all e.g. a network drive. If there will only be one user, it is more efficient to locate the output folder on the local machine or a USB 3 connected external drive. The Working Directory is where temporary files created during processing will be stored. The working directory is per computer/user. It is highly recommended that the working directory selected is on the local machine (or at least a USB 3 connected external drive) so that temporary files are not being created over a network or slow connection. If using USB 2 or a slow network connection, processes will run extremely slowly. Please ensure that this working directory is Page 27 of 78

28 writeable and has, at minimum, the size of the orthometric LAS dataset in available space. The working directory is wiped every time a QA session is initiated, therefore DO NOT select a directory containing other files or they will be deleted. If QA4LiDAR has been run previously, the working directory defaults to the last working directory used on that computer. The Mosaic Dataset option allows the user to point to the location of an existing ESRI DEM mosaic dataset residing within a geodatabase if one has been delivered with the project or pre-generated by the user from the project DEM tiles. This will save QA4LiDAR from needing to generate a DEM mosaic dataset, however if one does not already exist, this box can be left blank and the mosaic dataset generated later. If a mosaic dataset has previously been generated by QA4LiDAR for a project, and there has been no change to the project DEM tiles since its creation, the mosaic dataset directory will already be set to the mosaic. 4.9 CHECKS: Automated Checks The Automated Checks setup screen can be accessed under the CHECKS heading on the left menu panel. It allows users to supply Survey Control variables, Processing variables, and select the checks to be run. Certain information must be provided before the Automated Checks can begin. Page 28 of 78

29 4.9.1 Survey Control There are four Survey Control point types that can be input as shapefiles to QA4LiDAR; 1. Provider FVA Check Points (FVA CPs) o These are part of the Independent Check Point network supplied by the LiDAR provider and are used to assess the fundamental vertical accuracy of the survey. They must be gathered internal to the project area and are often collected in clusters on open, flat ground where there is a very high probability the sensor will have detected the ground surface, for easy comparison to LiDAR ground points. In QA4LiDAR they are used as part of the Survey Control Collection Method, Survey Control Density checks, the Survey Control Distribution check, as well as the Flatness, Openness & Absolute Vertical Accuracy checks. 2. Provider Ground Control (GC) o Along with Base Station data, GC (also supplied by the LiDAR provider) make up what is termed the Control Network. GC are high accuracy points (e.g. state benchmarks) that are used by the provider to establish the datum in the survey area and adjust the LiDAR horizontally or vertically. They can be internal or external to the project and assess the variation of the reference surface and/or geoid model across the survey area. In QA4LiDAR they are used for the Survey Control Collection Method, Survey Control Density checks, the Survey Control Distribution check, as well as for Flatness, Openness & Absolute Vertical Accuracy checks. 3. Custom (Purchaser control) o This can be any other control (or shapefile point elevation data of known accuracy) that the purchaser has access to. It will be used by QA4LiDAR for the Survey Control Density checks, the Survey Control Distribution check, and the Flatness, Openness & Absolute Vertical Accuracy checks. It should be internal to the survey area. 4. Provider SVA Check Points (SVA CPs) o These are part of the Independent Check Point network supplied by the LiDAR provider and are used to assess the supplemental vertical accuracy of the survey. They must be gathered internal to the project area and located in a range of different terrain types. In QA4LiDAR they are only used as part of the supplemental Absolute Vertical Accuracy check. *Note. The Survey Control Accuracy and Absolute Vertical Accuracy Checks can be run with just FVA Check Points, with both FVA Check Points and Ground Control, with just custom (purchaser) control, Page 29 of 78

30 or with all three of these control. If all three are supplied the checks will run once for FVA CPs and GC in combination, and separately for custom control. To add a Survey Control point shapefile, click the button beside the relevant type, navigate to the location, and Open the shapefile. Alternatively, type the location and a green tick will appear if the location typed can be found. Next, use the Elevation field (ORT) drop down list to select the field in each shapefile which represents AHD elevations of the control/check points. Do the same for Elevation Field (Ellipsoid) if applicable or select N/A. Note. It is recommended these fields in the shapefile be of type double. Finally, use the drop down list to select the Acceptability Rating you wish to use for the control/check points. This rating is used in the Flatness and Openness part of the Accuracy of Survey Control check and the Absolute Vertical Accuracy check. If the rating for Flatness or Openness is unacceptable (based on your acceptability rating) for a control/check point, that point is NOT used to test the accuracy of the Survey Control or the Absolute Vertical Accuracy of the data. BOTH Flatness and Openness for a control point must be rated equal to or better than the user s acceptability rating for the point to be used. The lower the rating selected (i.e. closer to 'Poor'), the more control/check points that will be used for that type. Refer to section 0 for the Flatness and Openness rating methods. Some testing of your control may be required i.e. you may want to start with a rating of Poor to run the Accuracy of Survey Control or Absolute Vertical Accuracy check, then examine the output control files for the Flatness and Openness rating results for each point, then try changing the rating to suit your requirements. The more points you supply, the longer these checks will take to complete. For SVA CPs, instead of the Acceptability Rating, the final drop down box is for the Cover Type Field. Select the field in your SVA CP shapefile that represents the land cover type to which each point is associated. E.g. you may have a filed called Class with attributes such as Tree, Grass etc Processing To add a Flightline Shapefile (polyline or polygon), click the button beside the box, navigate to the location, and Open the shapefile. Alternatively, type the location and a green tick will appear if the location typed can be found. Use the Point Source ID Field to select the field in the shapefile that represents the flightline number (integer as per the point source ID attribute in the LAS files). The flightline shapefile is used for the swath related LAS checks and is only required if your project has swath data, or if you want the results of the pseudo pulse density check output as attributes in a copy of the flightline shapefile. If your LiDAR data is bathymetric, supply a Bathymetry coverage tile index polygon shapefile which must be a 50m or 100m tile index, only covering the tiles to be included in the check (i.e. not shoreline tiles that are partially land). Also use the Bathymetry Classes box to type and add (click Add Class) the bathymetry point class/s to be used for the bathymetry coverage and relative vertical accuracy checks from the orthometric LAS. If your data is topographic, leave these sections blank and the default ground class 2 will be used for the relative vertical accuracy check. If your data is Page 30 of 78

31 topographic but you wish to include different classes in the relative vertical accuracy check you can override the default class by entering bathymetry classes i.e. class 1 and class 2 (or any relevant). The Raster Cell Size will be used to produce output rasters for the density checks as well as for data processing during the relative vertical accuracy check. If a cell size greater than 5m is selected, the relative vertical accuracy checks will NOT run. Use the slider bar to select the cell size you wish to use. The default (and minimum) cell size is 2m, while the maximum cell size is 10m. Note. Part of the relative vertical accuracy check process creates a boundary polygon of the data set with internal holes. If the raster cell size is set to 2m and there are a lot of internal holes in a large dataset, this output boundary polygon may exceed the maximum shapefile size of 2GB. To avoid this, increase the raster cell size to 5m. However, also note that as the raster cell size is increased, the area of this boundary polygon is slightly increased, so the final calculated accuracy result will be slightly lower Running Automated Checks The checks are organised into groups. The tick boxes can be used to select which automated check groups or individual sub checks you wish to run. The checks are listed in the order it is suggested they be run in. When a new project is opened, the Delivery Completeness* & Spatial File Corruption** check is the only one ticked. It is HIGHLY recommended that this check group be run FIRST without ticking any of the following check groups. If the delivery is incomplete or corrupt, it is likely subsequent checks will fail and significant time may be wasted running them. Hence if any part of this check group fails this will HALT the QA Session so that if you had more checks queued to run they will NOT run and SHOULD NOT be run until the failure issue is addressed. It is more efficient to update/fix any delivery incompleteness or corruption in the data before running other checks. For example, if any LAS files are found to be corrupt and they are left as corrupt in the project, they will cause some of the following checks to CRASH. Please un-tick the waveform and swath related checks (last 2 checks) within this group if those data types do NOT exist in your dataset as this will save time. *Note on delivery completeness. LAS are checked for delivery completeness in this group and other tiled datasets and their mosaics in the next check group as missing LAS halt the QA, while there may be legitimate reason for missing tiles for other data types. **Note on corruption. There are 6 parts to the corruption check for LAS files; file readability, number of points is not zero, file size, number of points is less than 100 per file, number of points is less than 100 per flightline, and maximum number of point source IDs. The lasinfo tool is used to check readability, number of points is not zero, and number of points per flightline which are the core corruption checks. If these core checks fails it returns the message Unable to open LAS file {file path}, it is corrupt or Error: {file path} contains 0 point records. The file is considered corrupt or Error: Flight line {x} contains less than 100 point records ({x} points). All files with this point source ID are considered corrupt.. In these cases, the user should remove these files or obtain new readable versions with points before running other checks or they will crash. The other three parts simply provide warnings to the user if the file size is greater than 2GB (ICSM specification requires LAS files <2GB), if the number of points is less than 100 (potentially causes processing issues), or if the number of point source ID s per file exceeds QA4LiDARs expected maximum. The maximum expected number of point source ID s is 10 times the tile size in km, which is to ensure these Page 31 of 78

32 numbers represent flight lines. If these warnings are given, the files can still be used in remaining checks. One exception to running Delivery Completeness & Spatial File Corruption check first, is if you only wish to run the Accuracy of Survey Control check to validate the proposed Survey Control in the planning stage of a project. After running the checks in Delivery Completeness & Spatial File Corruption successfully, it is a good idea to run both File Naming, Shapefile Attributes & Horizontal Coordinate System and Comparison of Report Form To Tender Form checks before queuing the remaining checks to run. If for example, there is a significant LAS or DEM file naming issue (i.e. tiles cannot be identified or AHD and ELL files cannot be separated), some of the remaining checks may be unable to run properly before this is addressed. It would be unfortunate if you had set all remaining checks to run over the weekend but upon return found they were unable to run successfully. Again please un-tick the swath related check (last check) within this group if that data type does NOT exist in your dataset as this will save time. When the first three checks have run successfully, the remaining checks can be run i.e. Classification Statistics, Accuracy of Survey Control, Point Density & DEM Resolution, Flight Line Coverage, and Absolute & Relative Vertical Accuracy. You may also select to Prepare Visual Checks which will generate the detailed DEM extent shapefile and produce the DEM mosaic for the project. Again, if you are working with a large dataset it may be wise to do this overnight or a weekend as it may take some time. There is no coupling between checks therefore checks are independent of each other. However, as some checks use bits of the same code and outputs, if these outputs have already been generated for one of the checks, subsequent checks will utilise them to save processing time. An example of this is the generation of LAS files by flight line which is used for the flightline coverage raster, the pseudo pulse density flight line rasters, and the relative vertical accuracy check. To run a check, tick the box for that check and click Run. Multiple checks can be ticked and queued to run sequentially. The Automated Checks QA Session running screen will appear and processing for the check/s will begin. For information on the methods used to run the checks, refer to Appendix 1. When a check is running, a blue circle will appear to the left of each check and a progress bar to the right as each check is processing. A disk icon will appear to the left of a check when the check is finished and the results are saving to the QA4LiDAR Project Database (or when previous results are being removed). There is a stop button on the upper right hand corner of the QA Session screen which allows you to abort the checks. There is also a back button that allows you to return to the Automated Checks screen. When the check is complete and saved, if a green tick QA. A yellow warning appears, the check has completed successfully but data has not necessarily Passed run for tiff files if no tiff files exist. A red warning run successfully. means the check cannot be run, for example a corruption check cannot Page 32 of 78 means an error occurred and the check did not

33 You can find additional information in the log window at the bottom of the screen. Each time a QA Session is run, the log is automatically saved to a text file in a folder called Logs in the output directory. Each log file is named with the date, time, and project folder name. When the QA Session has completed, you can click the Report screen to see the results of that check (refer to section 0). The QA4LiDAR QA Report information will be generated each time you click on Report and the output tile index shapefile will be updated only if a new QA Session has been run. Page 33 of 78

34 4.10 CHECKS: Extent Check After the Automated Checks have been completed, the user can progress to the Extent Check. If a DEM mosaic dataset was not provided on the Project Settings page, the user can either Return to Settings to provide an existing DEM mosaic dataset (supplied with the delivery or pre-generated from project DEM tiles), or choose to Generate Mosaic Dataset. If the user chooses to generate a mosaic dataset, a loading dialog will appear while the dataset is generated from the project DEM tiles. As the tiles are loaded into the mosaic they will colour green in the loading dialog. If a project is being re-opened, a mosaic dataset already exists and there has been no change to the project DEM tiles since its creation, this screen won t appear, as the existing mosaic will be used to avoid wasting time re-generating one. Once the mosaic dataset has been generated, the below continue screen appears. The user can either select Continue to load the Check Extents map using the basic extents, or they can tick the box to Generate detailed DEM extents (LAS extents will be basic either way - explained below). If you are working with a large dataset with many tiles, the basic extents can take time generate. The detailed DEM extent is optional as this can take a long time to generate for a large dataset. The same loading dialog as when generating the mosaic will appear, followed by the Check Extents map. Extents for the LAS and DEM data are generated while this loading dialog is visible. The LAS shapefile displayed in the Check Extents window is created by extracting the ArcGIS LAS Dataset LAS file extents. Hence, most polygons will be square (i.e. the tile size) while any boundary tiles partially filled with data will be rectangular (smaller than the tile size). The polygons will NOT show internal holes in the data (the flight line coverage raster could be analysed for this purpose). The basic DEM shapefile is extracted from the DEM mosaic footprint. Again, this extent will NOT represent internal holes. To identify internal holes in the DEM, the user should tick to Generate the detailed DEM extents (or alternatively, could analyse the Point Density & DEM Resolution automated check output density rasters). Page 34 of 78

35 The Check Extents map, shows the DEM boundary extent (hatched brown), the LAS boundary extent (hatched yellow), the original project extent (light blue outline), and the detailed DEM extent if generated (navy outline called No Data Extents circled in red in the image below). The detailed DEM extent only displays areas of no data in the DEM, so if it doesn t seem to appear on the map, your DEM is free of No Data holes. You are required to investigate the data and determine if the DEM and LAS extents cover the original project extent adequately, and if detailed DEM extents were generated whether there are any unacceptable voids in the DEM. You can turn layers on and off in the table of contents, zoom with the mouse wheel, right click a layer name in the table of contents and zoom to layer, and add data if required. The window will initially be zoomed to the extent of the data; hence if it does not appear to be zoomed to the project extent, there may be a coordinate system issue with one or multiple LAS or DEM tiles. If the extent of the data appears acceptable, select the relevant Yes radio button. If the project extent is not fully within the LAS and DEM boundaries, select the relevant No radio button (as per the below image), and briefly describe the problem in the box provided. If detailed DEM extents have been generated and any existing no data holes are acceptable, select the relevant Yes radio button (as per the below image). If any existing no data holes are unacceptable, select the relevant No radio button. Select Save and move to the Visual DEM Check. Page 35 of 78

36 4.11 CHECKS: Visual DEM Check Sampling The Visual QA Session screen can be accessed by going to the Visual DEM Check under the CHECKS heading on the left menu panel. It allows users to choose whether to visually check all DEM tiles or a sample of DEM tiles. There are two options available for selecting a sample. You can either load a pre-flagged tile index shapefile, or type the percentage of tiles to check. If typing a value, a random sample of tiles equivalent to the percentage entered will be highlighted on the tile map. If you wish to check all DEM tiles, type the percentage as 100. Two forms of intelligent sampling based on the user s knowledge of the survey area are enabled; the first is the pre-flagged tile index and the second the ability to manually select/unselect tiles on the map. Zoom tools are available to assist the second option. The user may want to target tiles in coastal areas, with steep terrain, known water bodies, or heavily vegetated areas. If a user manually selects/unselects tiles, the sample percentage is updated to match. To use the first option and load a pre-flagged tile index, the user can browse for a polygon shapefile of pre-flagged tiles and select the flag field. The flag field should be an integer field of 0s and 1s where 0 are tiles you don t want to sample and 1 are tiles you do want to check as part of your sample. The import of these flagged tiles is a once off update of the sample. The user is then able to manually change the sample tiles by clicking them however will not be able to revert to the imported selection unless they re-import the pre-flagged tile index. Only those tiles in the pre-flagged index that actually match the project tile index and that have not already been selected for sampling by another user will be selected. If a tile index that does not match the project tile index is imported, a warning will be displayed. To enable pre-flagged tile selection for multiple users, you are required to create and import a preflagged tile index per user with the relevant (different) tiles flagged, or import the same tile index multiple times using different pre-flagged fields for each user. For multiple users, tile sample selection is based on the user information. Tiles in the sample selection will be coloured by user. Users will not be able to change the sample selection of other users. If a user/s has already been allocated tiles to check, and a new user selects to sample 50% of tiles, they will be allocated 50% of the remaining tiles available. This is to avoid tile collision i.e. two people checking the same tile. To perform Visual Checks on re-delivered DEM data the user would want to target only the DEM tiles that had been re-delivered. As mentioned, the DEM mosaic will be regenerated if DEM tiles have changed, however the system will not automatically recognise which tiles these were as part of the sample selection. The solution to this is to load the tile index with the first set of Visual Check results and use the failed tiles as the flag for sample selection (refer to section ). When you are happy with your sample selection click Continue. Page 36 of 78

37 Multiple Users QA4LiDAR supports multiple users for the Visual Checks if the project data and output folder are located on a network drive that multiple users can access. The speed of your network will determine the responsiveness of the Visual Checks when there are multiple users. The conventions for Visual Checks with multiple users are as follows; Only one user can run the Automated Checks at a time The Visual Checks cannot be run while the Automated Checks are running Multiple users can perform Visual Checks at the same time The project data and output folder must be on a network drive accessible by all users The Visual Checks step each user through the tiles in their sample selection (unless they manually navigate off this sample) and they display all tiles checked by all users with the green checked border QA Sessions have an expiry time in case of crash so you don t get locked out The Dashboard displays any Active Sessions that are running The Visual QA Session sampling selection screen displays the tiles allocated to each user The Visual Checks Minimap displays the tiles checked by each user The name of the user who checked each tile is written to the output tile index Page 37 of 78

38 Check DEM Tiles The Visual Checks screen appears next. If it seems to freeze part way through loading, click the refresh button. It consists of a table of contents, a menu panel, and the map window. The DEM mosaic is displayed in the table of contents and map window using bilinear interpolation for display. QA4LiDAR zooms to the first tile to be checked, which is outlined with a black square and divided into quadrants. It also displays a portion of the surrounding tiles so that boundary issues and cross tile errors can be identified. Other tiles are outlined with a grey border. There is a Refresh button to refresh the Visual Checks screen in case it slows down or freezes. If you zoom out too far the DEM won t display. The following section explains how to use the Visual Checks interface to check DEM tiles, including a table of shortcut keys (section ) to use for efficiency. Table of Contents Menu panels Map window There are a number of ways to navigate through the tiles. The Previous and Next buttons in the menu panel can be used, or the shortcut keys for these which are f and g respectively. Alternatively, clicking Mark as Checked will move you to the next tile or the shortcut key for this which is spacebar. It is also possible to click on part of one of the visible surrounding tiles to move to that tile or use the arrow keys to navigate to surrounding tiles. Scrolling the mouse wheel zooms in and out to change the scale. There are also Zoom buttons which allow fixed zoom in, fixed zoom out and fit to tile. When checking a sample of tiles using the Previous, Next and Mark as Checked forms of navigation, QA4LiDAR will step you through the sample. If the manual forms of navigation are used i.e. clicking on a surrounding tile or using the arrows to navigate, the user may navigate off the sample. However if they go back to using Previous, Next and Mark as Checked they will be taken back to checking the sample selected. If the user selected sample tiles on the Visual Checks initiation screen manually, navigation is in order of tile selection. If you are reviewing someone else s Visual Checks the buttons Next Error and Clear Errors are useful. The Next Error button will take you to the next error that has previously been marked up so you can Page 38 of 78

39 check if the mark up is correct. If it is not correct you can use Clear Errors to clear the error in a particular tile quadrant or all errors within the tile. The DEM symbology can be changed using the Colour Ramp button. When clicked, a symbology panel appears on the right hand side from which the user can select from a range of standard symbology, including elevation symbology (Elevation #1 and Elevation #2). There is a toggle box at the top to Stretch to Display Extent so that greater colour variation can be seen in the tile. It is also possible to Set NODATA Colour at the bottom of this panel. You may wish to set this to a contrasting colour to the symbology so areas of no data can be easily identified (e.g. pink as below). The chosen symbology is retained as part of the user settings profile so that when the user navigates away from and back to the Visual Checks their setting remains. Error mark up for DEM tiles is broken into 3 parts; location (tile quadrant), error type (relative vertical accuracy, classification, interpolation, systematic errors, follow up, and other), and error size (small, medium, or large). These qualities should help identify the errors later on when referring to the tile index. To check each DEM tile for errors, examine the tile and if an error is found, right click on the quadrant containing the error or use the shortcut keys 1, 2, 3, 4 which apply to the quadrants as below. The quadrant will be highlighted red and an error dialog will appear. The error dialog can be repositioned if required or the ESC key used to exit it Page 39 of 78

40 To mark the error/s, select the appropriate error type/s and size using the mouse or the shortcut keys q, w, e, r, t, y for the errors and a, s, d for the sizes. Then select Save. Errors can be removed by selecting Cancel or the ESC key instead of save or later by right clicking the quadrant again and un-selecting the error/s. Multiple error types can be recorded against a tile and/or quadrant although only one size of error per quadrant. When finished checking a tile click Mark as Checked or use the spacebar key which will change the tile boundary from black to green to signify that it has been checked and move you to the next tile. For tiles that have already been checked (green boundary) you can also Mark as Unchecked or use the spacebar key which will turn the boundary back to black but NOT remove the errors (use Clear Errors) or move you to the next tile. The errors are written to the output tile index shapefile upon generation of the QA Report. There is also a Screenshot button to capture a PNG format image of the current tile, which is automatically saved to a folder called Screenshots in the output directory. The screenshots are named with the user, date, time and tile number so they can easily be identified. They may assist in reporting DEM errors to the provider. If you have chosen to check a sample of tiles and are finding a lot of errors in the sample, you can opt to Change Sample and increase the percentage of tiles you are checking. When you click the change sample button you will be taken back to the Visual Checks setup screen where you can adjust the sample percentage and continue back to the Visual Checks screen. It is possible to produce a hillshade of the DEM to assist with error identification. Click the Hillshade button in the menu panel. The Hillshade dialog asks you to specify the Azimuth, Altitude, and Z Factor you wish to use to create the Hillshade. The default values are the defaults used by ESRI. When you Generate the Hillshade, it will appear in the table of contents and map window and be saved to the output directory in the folder of the user who produced it within the VisualChecks folder. The symbology for the Hillshade cannot be changed, however you can turn its display on and off using the table of contents tick box. Page 40 of 78

41 To further assist error identification, it is possible to add other data, namely aerial photographs using the Add Data button. The intention here is NOT to replicate ArcGIS. The add data button is primarily intended for aerial photographs. Spatial Database Engine (SDE) is NOT supported. If vectors are loaded their symbology cannot be changed. A workaround for these things is to use layer files. You can also use the View Profile button to draw a profile on the DEM in the map window. It is possible to draw a single straight line profile as shown below, or a multi-line profile with vertices. Only the start point A (the left of the profile graph), and end point B (the right of the profile graph) will be marked. Once the line is drawn, the profile graph appears at the bottom of the map window. Units are in metres. The Hide button can be used to close the profile. The progress of checked tiles can be viewed using the Minimap. The Minimap displays the tile index in grey outline and colours the tiles (in a different colour for each user) when they have been marked as checked. If you hover the mouse over tiles in the Minimap, you can see which user has checked each tile. The Minimap can be printed to PDF using the Print button in the Minimap window. Page 41 of 78

42 Error Mark up Tile Navigation QA4LiDAR version User Manual To exit the Visual Checks screen there is a Finish QA button which moves you back to the Dashboard. When you return to the Visual Checks, the user settings are retained. Hence if you have the same user settings set, you will be taken back to the tile you were checking, the symbology will remain as set, and any loaded data will persist. In order to see the results of the Visual Checks in the output tile index, the user MUST generate the QA4LiDAR Report so that the tile index is updated Shortcut Keys Table 3 outlines the shortcut keys that are available for the Visual Checks. They are intended to allow the majority of the DEM checking to be done via the keyboard to make the process efficient. Table 3. QA4LiDAR Visual Check shortcut keys Action Shortcut Key Previous tile f Next tile g Left tile arrow key Above tile arrow key Right tile arrow key Below tile arrow key Change scale scroll mouse wheel Mark as Checked spacebar Mark as Unchecked spacebar Open error dialog for NW quadrant 1 Open error dialog for NE Quadrant 2 Open error dialog for SW Quadrant 3 Open error dialog for SE quadrant 4 Escape from error dialog ESC Relative vertical Accuracy error q Classification error w Interpolation error e Systematic error r Follow Up error t Other error y Small size error a Medium size error s Large size error d Save error/s Enter Page 42 of 78

43 4.12 OUTPUT: Report The QA4LiDAR QA Report can be accessed under the OUTPUT heading on the left menu panel. When the Report is clicked, the messages Updating tile index shapefile... followed by Generating Report Data and Report Data Generated appear while the report is generated and tile index updated with the latest check results. The tile index will only be updated once for every QA Session run. The Report can be generated in sections, after each check is successfully run. You can run one or multiple checks at a time and see the results for those checks on the QA Report without having to run all checks. The QA4LiDAR and ArcGIS versions are printed at the top of the report, along with the project Contract Number and Project Title which are extracted from the Tender Form, and the Project Directory. If no forms exist, the contract number and project title will be blank. A date and time stamp is also printed at the top right of the report every time it is generated. Before a check has completed, the Compliance for that check will state PENDING. The types of compliance stated on the Report are explained at the top of the report. The user is able to change the compliance found by QA4LiDAR by hovering the mouse over the particular compliance and using the drop down box that appears. For example, a user may change compliance from FAIL to CPASS if the data was very close to passing and is deemed acceptable. If a change is made, an asterisk will appear next to the new compliance value meaning * : The compliance value has been changed by the user and is different to the value originally assigned by QA4LiDAR. If a check has failed, the element will be highlighted in red. The user can left click on the highlighted/failed element for more information about the failure or to comment on the result e.g. why it was changed from FAIL to CPASS. Left clicking on any non-red element will allow a comment to be made if required. The detailed fail information is saved to a QA Errors PDF, while any comments are printed under the relevant results table on the QA Report PDF. Changes to compliance and any comments added are saved, unless the check is re-run, in which case any manual changes are overridden. Page 43 of 78

44 By hovering over any entry in the left hand column of each table, a tool tip will be displayed which gives a brief explanation of what each check does. For more information on how the checks are performed, see section 9.1. The sections of the Report are presented in the same order as the suggested running order of the Automated Checks (as listed on the Automated Checks screen). The results in each section are explained under the relevant headings below. If a user wishes to re-run a set of checks, they may click the blue button directly beneath the check results which will return them to the Automated Checks screen with the relevant checks ticked. All the user needs to do then is click Run, and the checks will be re-run. Users should be aware that existing results for a check will be overwritten if the check is re-run Presence & Reading This section displays results for the Delivery Completeness & Spatial File Corruption and File Naming, Shapefile Attributes & Horizontal Coordinate System checks. The Check column states the check performed, the File Type column states the data format the check is applicable to where possible, the Compliance column gives a PASS/FAIL statement, the Problem column states why the element failed (it can only display one problem per check), and the Number of Failed Files column states the number of failed files out of the total number checked. Any file types not applicable to the project are removed from the results. The green button appears below the table and links to the output location of the tile index shapefile. This is a copy of the original tile index with many additional fields populated with QA results. If any row in the table is left clicked and comments recorded, the comments appear below the table for the check and format and are colour coded to match the compliance. Comments can be left for any row in any of the results tables. Page 44 of 78

45 Forms Report This section displays results for the Comparison of Report Form to Tender Form checks. The Form Element column states the elements of the forms checked, the Compliance column gives a PASS/FAIL statement based on whether the information in the two Forms match, the Specified column states the value found on the Tender Form, the Reported column states the value found on the Report Form, and the Additional Corrections column is only applicable to the Geoid Model. If there were no Forms supplied to QA4LiDAR, these results will all be recorded as N/A. The section also displays a PASS/FAIL for whether the supplier certified the form for the delivery. Page 45 of 78

46 Classification Statistics This section displays results for the Classification Statistics checks. There may be up to 4 tables in this section depending on the data in your project. The main table consists of results for ORT LAS (orthometric) data. The Required/Unwanted column states the classes that were highlighted green (Required), red (Unwanted) or white (Ignored) on the Tender Form, the Compliance column gives a PASS/FAIL statement based on the existence of these classes in the classified ORT LAS data, the ORT Point Class column states the class number and description, the Point Count column gives the number of points found in each class, the % Points column gives the percentage of points found in each class out of the total, the Z Min column gives the minimum elevation value found in each class, and the Z Max column gives the maximum elevation value found in each class. Ignored classes, which are those not specifically required or unwanted in the dataset, are reported with N/A compliance. If your project contains ellipsoid LAS data there will be a second results table for this. It consists of the same fields except the Point Class field is named Ellipsoid Point Class. The compliance column gives a PASS/FAIL statement based on whether the Point Count for each class matches the Point Count for the same class in the ORT LAS results. If your project contains ellipsoid swath LAS files there will also be a results table for this data. It will only have entries for each class that is present in the data. As there should only be class 0 points in unclassified swath LAS data, class 0 will PASS and any other classes present will FAIL. Finally, if there is more than one type of LAS file in your project, there will be a comparison table showing the total point count for each LAS dataset and providing a PASS if the point counts match, or a FAIL if the point counts differ. There should be the same number of points in each LAS dataset. Page 46 of 78

47 Survey Control This section displays results for the Accuracy of Survey Control checks. It is split into two tables. In the first table, the Check column states the check performed, the Compliance column gives a PASS/FAIL statement based on the minimum number of required points for ICSM Density (PASS/FAIL results are not applicable for QA4LiDAR Distribution), the FVA Check Points Found (Minimum Required) and Ground Control Points Found (Minimum Required) give the number of points found followed by the minimum required in brackets (based on standard ICSM requirements), and the Survey Control Rating column provides the QA4LiDAR rating given by the check. *Note. The Pass/Fail result for the ICSM Density check are based on the ICSM minimum point requirements, whereas the Control Density Rating is based on QA4LiDAR s own rating system as described in Appendix 1. As the ICSM minimum requirements are quite high, it is possible for the density to FAIL with a strong rating. In such a case, the user may opt to conditionally pass the Survey Control. In the second table, the Delivery Element column states the type of control point, the Method column states the method used to collect ground control as reported on the Report Form, and the ORT Connection column states how the ORT connection was established as reported on the Report Form. The user needs to decide whether these explanations are acceptable. Page 47 of 78

48 Density / Resolution This section displays results for the Point Density and DEM Resolution checks. It is split into three tables, the first of which is for LAS point density results. The point type is stated in the Delivery Element column, the Compliance column gives a PASS/FAIL statement based on a comparison of the Required NPS column whose value is obtained from the Tender Form and the calculated Pseudo Pulse Density column result (the average result for all flight lines). Also reported are the calculated All Point Density and Ground Point Density and the Number of Failed Flight Lines (as the compliance is for Pseudo Pulse Density which is calculated per flight line). The pseudo pulse density value for each flightline can be found in the output flightline shapefile if an input shapefile was provided. The second table is for DEM resolution results stated in the Delivery Element column, the Compliance column gives a PASS/FAIL statement based on a comparison of the Required Resolution column whose value is obtained from the Tender Form and the Found Resolution(s) column. The number of Failed Tiles out of the total number of DEM tiles is also reported. If there are multiple DEMs of different resolution in the delivery, the DEMs that don t match the required resolution will appear as FAILs. This can be adjusted to a CPASS. The third table is for bathymetry coverage results. The point type is stated in the Delivery Element column, the Compliance column gives a PASS/FAIL statement based on a minimum % of tiles having a minimum number of soundings as requested on the tender form. Also reported are the Required Minimum Soundings, the Required Minimum % of Tiles, and the % Results or percentage of tiles having the minimum required number of soundings as found by QA4LiDAR. Green buttons and appear below the tables when the checks have been successfully run. When clicked, these links open the folder containing the output ground point density raster and all point density raster. These output rasters are generated at the cell size chosen on the Automated Checks set-up screen and therefore density values within the rasters are NOT per square metre. They are per square cell value chosen (default and minimum is 2m). They give a good indication of the variation of density across the project and can be further processed by the user outside of QA4LiDAR to achieve density per metre squared if desired. They can also be used to find internal voids in the data. Page 48 of 78

49 Flight Lines This section displays results for the Flight Line checks. This check only applies to the classified ORT LAS points stated in the Delivery Element column, the Compliance column gives a PASS/FAIL statement based on a comparison of the Required Relative Vertical Accuracy value which comes directly from the Tender Form and the Relative Vertical Accuracy Statistics value which is calculated by the check. If there is <2% flight line overlap, the check will not run and the compliance will be N/A. The second table is for the scan angle results for classified LAS as stated in the Delivery Element column. The Compliance column gives a PASS/FAIL statement based on a comparison of the recorded scan angles within the LAS point data to the allowable Maximum Scan Angle value which comes directly from the tender form. The Number of Failed Tiles out of the total is also given. A green button appears below the table when the Flight Line Coverage part of the check has been successfully run. When clicked, this link opens the folder containing the output flight line coverage raster. The flight line coverage raster output is generated at 1m cell size (based on the pseudo pulse density flight line rasters which are required per metre) so may appear a bit pixellated however can be useful in highlighting internal voids in the data Vertical This section displays results for the Absolute & Relative Vertical Accuracy checks. The Delivery Element column states whether the check has been performed on the LAS or DEM, the Survey Control column states which Survey Control has been used in the check, the Compliance column gives a PASS/FAIL statement based on a comparison of the Required Absolute Vertical Accuracy column value which comes directly from the Tender Form and the Absolute Vertical Accuracy Results column value which is calculated by the check. The Acceptability Rating that the user selected for each type of control on the Automated Checks set-up screen and an Acceptable FVA Check Points Count (i.e. the number of FVA Check Points used in the absolute vertical accuracy check) are also stated. If supplemental check points were supplied, a similar results table exists for them. However, there are no Compliance, Required Absolute Vertical Accuracy, Acceptability Rating, or Acceptable FVA Check Points Count fields as these are not relevant. There is a new Land Cover Type column which states the attribute from the class field i.e. Tree, Grass etc. that the calculated accuracy relates to. Page 49 of 78

50 A green button appears below the table when the check has been successfully run. When clicked, this link opens the folder containing the output FVA Check Points and Ground Control shapefiles. These are copies of the original files with four additional fields; Table 4. Survey control output shapefile fields added Field Name OPENNESS FLATNESS LAS_DIFF DEM_DIFF Description User acceptability rating for openness User acceptability rating for flatness The height difference between LAS and each control point in metres The height difference between LAS/DEM and each control point in metres *Note. If the user checked the accuracy of the survey control before acquisition (i.e. LAS data were not present), the OPENNESS and FLATNESS fields are generated in the output control shapefiles and all populated with the value STRONG. These values of strong are not accurate ratings; they are simply for programming purposes so that all points are used in the Accuracy of Survey Control check Visual Checks (DEM) This is the only section of the report for which the compliance and comments are not automatically updated or re-set when visual checks are re-run. The section displays results for the Extent & Horizontal Coordinate System, and Visual checks. The first two tables provide the results of the visual extent checks; first whether the data extents were deemed valid and if not, the details of the issue, and second if detailed DEM extents were produced, whether any internal voids are acceptable or not. The third table is the results of the Visual DEM checks. The DEM Error Type column states the error type found, the Compliance column states PASS unless tiles have been found with an error in which case it will be FAIL, and the Number of Errors is also reported. You can click any FAIL lines to see the tile name and quadrant with the error. Page 50 of 78

51 Output Supporting Information A number of supporting data files are generated by QA4LiDAR and saved to the Output Folder chosen by the user. After the data has been generated, the links on the QA Report turn green and when clicked, open the folder location of the data. Links also appear on the Dashboard. The user can then open the data in a GIS. Within the output folder chosen, QA4LiDAR generates a folder with the same name as the project. Within this project output folder are the following; Logs folder QA4LiDAR.Checks folder QA4LiDAR.References folder Screenshots Visual Checks folder Tile Index shapefile The Logs folder contains the text log files automatically saved when automated checks are run. The files are named with the date, time and project name. One log file is saved per QA Session run. The QA4LiDAR.Checks folder contains a number of spatial output files generated by the checks including; survey control points with additional fields added by QA4LiDAR as part of the flatness and openness check (in the PointStatistics folder and AbsoluteVerticalAccuracy folder), some of the datasets created in the Survey Control check (convex_hull, difference, extent_dissolved, and extent_mean_center in the ControlDistributionStatistics folder), the flight line coverage raster ( FlightLineCoverage folder), the point density rasters and pseudo pulse flightline results ( PointDensity folder), the bathymetry coverage results ( BathymetryRaster folder) with the results field being GRIDCODE in the shapefile, and the flight line area polygon ( RelativeVerticalAccuracy folder). Not all processing dataset are output to the user as some are created in memory. The QA4LiDAR.References folder contains the flight line rasters produced by the Point Density check which are used by the Flight Line Coverage check. The Screenshots folder stores the PNG screenshots captured in the Visual Checks part of the software. Images are stored named by user, date and time. Page 51 of 78

52 The VisualCheck folder stores the extents, DEM mosaic dataset and user related information used by the Visual Checks so they do not have to be regenerated. The copy of the Tile index is saved directly to the Output Folder with the original tile index name. The fields in the output tile index include; Table 5. Tile Index output shapefile fields added Field Name CQTILENAME map_code corrupt naming hcs las_header scan_angle den_all_pt den_grd_pt cls_x_pts cls_x_min cls_x_max set_lasahd set_lasell set_dem set_dsm set_rgb set_int_a set_int_f set_int_l err_q1 err_q2 err_q3 err_q4 reviewer Description A QA4LiDAR generated tile name field (from the tile index polygon locations) The code used to display the colours in the QA map refer to the map legend The per tile Pass/Fail result for the corruption check The per tile result for the naming check with any dataset that failed listed The per tile result for the HCS check with any dataset that failed listed The per tile Pass/Fail result for the LAS header check The per tile Pass/Fail result for the scan angle check The per tile results for all point density The per tile results for ground point density The number of points in the class x within the tile, only classes present in the dataset are represented The minimum Z value in the class x within the tile, only classes present in the dataset are represented The maximum Z value in the class x within the tile, only classes present in the dataset are represented If AHD LAS were requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If ELL LAS were requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If DEM was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If DSM was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If Imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If all return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If first return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* If last return intensity imagery was requested, whether each tile for that dataset was Supplied, Multiple Supplied, Not Supplied or the check remains Pending/Not Supplied* The per tile quadrant 1 results for the Visual Checks** The per tile quadrant 2 results for the Visual Checks** The per tile quadrant 3 results for the Visual Checks** The per tile quadrant 4 results for the Visual Checks** The user name of the person who checked the tile with an error Page 52 of 78

53 *There is no easy way of telling whether 0 results is due to a pending check or 0 files existing, hence "Pending/Not Supplied". If there is at least 1 file present for a certain delivery element, all subsequent tiles will correctly be reported as "Not Supplied". **A cell is attributed with comma separated numbers that represent the error types, plus a letter for error size e.g. err_q1 = 2,5,M means quadrant 1 of that tile has classification and follow up errors of medium size. The codes are as follows; o o o o o o o o o 1 = Relative Vertical Accuracy error type 2 = Classification error type 3 = Interpolation error type 4 = Systematic error type 5 = Follow Up error type 6 = Other error type S = Small size error M = Medium size error L = Large size error Printing the QA Report When all necessary results are displayed on the QA Report and any applicable compliances changes have been made, the QA Report can be printed to PDF. It is possible to hide sections of the QA Report that may not be applicable to the project. To hide a section left click on the blue heading bar. Any hidden sections will not appear in the printed PDF. To Print the QA Report go to the File menu Print Report Summary. A Save As dialog will appear for you to choose the output directory and file name. Once the PDF has been created, you can open it in Adobe to print a hard copy. The PDF lists the sections of the report as bookmarks in the left panel for easy navigation. It is also possible to print a detailed list of all the QA errors/fail results. To do so go to the File menu Print QA Errors. Page 53 of 78

54 4.13 OUTPUT: Map The QA results are also represented visually in the QA Map. It highlights whether tiles were delivered and passed or failed automated and visual checks, providing a quick and basic visual review. If you require a more detailed view of the QA errors results as they apply to the tile index, please add the tile index to ArcMap (or a GIS) and review it there. Tiles in the QA4LiDAR QA Map are coloured according to the below flow chart which can also be found via the software Help menu under Map Legend. Page 54 of 78

55 5 Common Tasks 5.1 Approving the survey control design The ICSM Template states that The proposed check point survey design must be submitted with the quotation, and approved by the Contract Authority prior to implementation. The Contract Authority can use QA4LiDAR for this approval if they obtain shapefiles of the proposed Survey Control (Ground Control and FVA Check Points) and run the Accuracy of Survey Control check in the planning stage of the project. To do so; 1. Create a pseudo project folder to scan into QA4LiDAR containing the project extent shapefile, the ground control point shapefile and the FVA check point shapefile. 2. In QA4LiDAR create a New Project and choose the pseudo project folder you created to scan in. 3. On the Project Settings screen, provide the Project Extent shapefile, an Output Folder and a Working Directory. Remaining elements can be left blank. 4. Proceed to the Automated Checks screen and supply the FVA Checkpoints shapefile and Ground Control shapefile. There is no requirement to provide elevation fields or acceptability ratings for the control files for this task. 5. Tick only the Accuracy of Survey Control check group and select Run. 6. When the check has run, go to the Report screen and scroll down to the Survey Control section. Hide all other areas on the report and print to PDF. *Note. The Pass/Fail result for the ICSM Density check are based on the ICSM minimum point requirements, whereas the Control Density Rating is based on QA4LiDARs own rating system as described in Appendix 1. As the ICSM minimum requirements are quite high, it is possible for the density to FAIL with a strong rating. In such a case, the user may opt to conditionally pass the Survey Control. **Note. The OPENNESS and FLATNESS fields generated in the output control shapefiles in this case are all populated with the value STRONG which is not an accurate rating as there are no LAS files within the project. It is simply done for programming purposes so that all points are used in the Accuracy of Survey Control check. Page 55 of 78

56 6 Troubleshooting To prevent errors occurring while running QA4LiDAR, please follow the requirements and recommendations outlined in section 0. If an error does occur, the error message and information provided in the QA Session log file and QA Report will help identify the problem. Some investigation of the project data causing the issue may be required. If no problem with the data can be identified, try exiting and restarting QA4LiDAR and your computer before running the check again. Ensure project files are not open in other software, and close any instances of ArcGIS open on the computer while QA4LiDAR is running. You may also try rescanning your project directory or renaming/deleting the QA4LiDAR Project Database.cqp and clearing the output folders and starting again. If the error still occurs, contact the CRCSI to report the problem. Any updates to QA4LiDAR and the QA4LiDAR Form Editor will need to be supplied by the CRCSI. These can be downloaded and installed as required and will provide bug fixes and new functionality. Things to watch out for when running QA4LiDAR; Ensure the Requirements and Recommendations for using QA4LiDAR as outlined in section 0 have been followed. Ensure the computer in use has continuous access to the ArcGIS license and extensions. Ensure the computer in use is not set to hibernate after a period of time or overnight while QA4LiDAR is processing. Ensure project datasets are tiled as per the ICSM specification i.e. origins that align with the zero origin of the MGA Zone, on a whole metre coordinate value of the south west corner of each tile. The ESRI error type of "Error accessing tool..." (as seen in the QA4LiDAR log file), is due to ArcGIS not releasing a Geoprocessing tool correctly. If a check runs into this error, subsequent checks within the same QA Session are likely to encounter the same error. o It can be overcome by closing QA4LiDAR, re-opening the project, and re-running the check. The Unexplained ESRI operating system error (as seen in the QA4LiDAR log file which is ESRI error 99999) is an operating system error or generic ESRI error for which the cause is unknown. o It can be overcome by closing QA4LiDAR, re-opening the project, and re-running the check. Ensure large datasets are split into manageable chunks e.g. of ~2,000 x 1km tiles. *Note. An ESRI ArcObjects bug has been found with ArcGIS version (no service pack) in which ecw files are deemed corrupt despite being able to be opened and viewed fine in ArcMap* To fix, please update ArcMap to Page 56 of 78

57 Survey QA4LiDAR version User Manual 7 Future Improvements Future improvements may be made to QA4LiDAR by implementing additional checks to extend the capability of the software and by altering programming methods to improve efficiency. In addition, if changes are made to the ICSM Template, QA4LiDAR should be adjusted in line with these changes. For example, changes to the way in which point density should be measured, or changes to the understanding of the use of overlap points etc. It may also become relevant for QA4LiDAR to be applicable in New Zealand and hence incorporate the differences in their acquisition template. 8 Glossary Term Description Example Absolute Vertical Accuracy Aerial Imagery Also known as Fundamental Vertical Accuracy is the vertical accuracy in open terrain tested to 95% confidence (normally distributed error) of the elevation data when compared to survey control points (FVA Check Points and Ground Control). Or Aerial Photography can be coincident or noncoincident orthorectified imagery in 3 (RGB) or 4 (RGB + infrared) bands. <= +/- 30cm. 95% confidence interval (1.96 x RMSE) Bilinear Interpolation Linear interpolation performed in two directions (e.g. X & Y) so as to be a distance weighted average of the four nearest values. Classification Level Contours LiDAR point classification Levels 0 to 4 - refer to pages of the ICSM LiDAR Acquisition Specifications and Tender Template Version 1.0, November Vector representation of topography/bathymetry where continuous curved lines represent constant ground height values above a certain datum e.g. AHD. Level 0-4 Control Density Control Distribution Ground Control (GC) The number of FVA Check Points and Ground Control in the project area. QA4LiDAR uses minimum number requirements as well as a rating determined by the example formula (e=density, n=number pts, a=area). The spread of FVA Check Points and Ground Control across the survey area. QA4LiDAR uses a three part rating system (see Appendix 1). High accuracy GC (e.g. state benchmarks) are used to establish the datum in the survey area. They can be internal or external to the project and assess the variation of the reference surface and/or geoid model across the survey area. e = n a Page 57 of 78

58 FVA Check Points (CPs) Digital Elevation Model (DEM) FVA CPs are used to assess the vertical accuracy of the survey. They must be gathered internal to the project area and are often collected in clusters on open, flat ground for easy comparison to LiDAR ground points. A raster representation of the topography/bathymetry where cell values represent ground heights above a certain datum e.g. AHD. Digital Surface Model (DSM) Ellipsoid Environmental Conditions Flatness Flight Line Coverage Flight Trajectory A raster representation of the earth s surface including objects such as trees and buildings where cell values represent object heights above a certain datum e.g. AHD. Or reference ellipsoid is a mathematically defined surface that provides a simplified approximation of the geoid for coordinate system definition. There may be certain environmental conditions imposed for a LiDAR data capture such as; Cloud and fog free between the aircraft and ground Floodplain/wetland data must be captured during times of base-flow and outside of significant surface inundation due to natural events and /or regulated environmental flows Coastal surveys (areas under tidal influence) should be flown within 2 hours either side of low tide to minimise the effect of standing water or wave action Flights should not be undertaken during periods of heavy smoke haze How flat the area around each control point internal to the project area is. QA4LiDAR uses the heights of surrounding LiDAR ground points to determine this. How well the survey area is covered by the flight lines. There should not be any gaps between flight lines due to the aircrafts flight path. Or flightlines describes the aircraft flight path, usually represented by a line shapefile. GRS80 Geoid Model Horizontal Coordinate System The geoid is an equipotential surface (surface to which gravity is always perpendicular) that coincides with mean sea level (if at rest). A geoid model is an irregular 3D representation of the geoid which defines zero elevation and is used as a surface from which to measure elevations. A system which allows determination of the horizontal position of a point on earth. Page 58 of 78 AUSGeoid09 GDA94

59 Point Density QA4LiDAR version User Manual Intensity Imagery LAS LAS Dataset LiDAR An image created from the LiDAR intensity values i.e. the return strength of the laser pulse for every point, which looks like black and white aerial photography. It is based in part on the reflectivity of the object struck and is a substitute for aerial imagery when none is available. The common LiDAR data exchange file format refer to the ASPRS format specifications. A LAS Datasets is an ESRI/ArcGIS format that stores reference to one or more LAS files on disk. Light Detection and Ranging remote sensing technology that measures distance using a laser and produces a point cloud of elevation data. /Committee- General/LASer-LAS- File-Format- Exchange- Activities.html National Elevation Data Framework (NEDF) naming convention Nominal Pulse Spacing (NPS) Openness Orthometric (ORT) All Point Density Ground Point Density First (or last) return point density Developed to provide easy ingestion into the NEDF- Portal refer to pages of the ICSM LiDAR Acquisition Specifications and Tender Template Version 1.0, November The target number of outbound LiDAR pulses over a given area set prior to data collection. As unsuccessful pulses can t be measured, this is simulated in QA4LiDAR using last return and excluding data gaps to get a measure of Pseudo Pulse Density. How open the area around each control point internal to the project area is. QA4LiDAR uses the classifications of surrounding LiDAR points to determine this. The curved-line distance following the earth s gravity field from the geoid to the point of interest. Orthometric heights can be used to predict and measure direction and rate of fluid flow. The number of successful ground and non-ground point returns (1st, 2nd, 3rd AND last return) over a set area (e.g. more points returned in vegetated areas due to the presence of 2nd & 3rd returns). The number of successful ground point returns (1st, 2nd, 3rd OR last return) over a set area, which equates to removing all non-ground points from the point density (e.g. a typical ground point density required to generate a DEM is 2 points per square metre). The number of successful 1st (or last) returns over a set area which could be ground or non-ground (e.g. only by examining first or last return or pulse density will you find areas of greater density in a project). Page 59 of 78 au/elevation/lidar_s pecifications_and_te nder_template.pdf 2 AHD (normalorthometric) 2.42 pts/m pts/m pts/m 2 Points at The number of successful ground and non-ground point 2.87 pts/m 2

60 Nadir Pulse density / Pseudo Pulse Density Relative Vertical Accuracy returns (1st, 2nd, 3rd AND last return) over a set area at nadir (middle 10% of swath width). The number of outbound pulses (not necessarily successful returns) over a set area. This is simulated in QA4LiDAR using last return and excluding data gaps to get a measure of Pseudo Pulse Density. The accuracy of LiDAR data between flight lines i.e. how well the flight lines align with each other. Resolution The grid cell size of the DEM generated from LiDAR. 1m Root Mean A measure of the differences between values predicted Square (RMSE) by a model or an estimator and the values actually Scan Angle Supplemental Vertical Accuracy Tile Index Triangulated Irregular Network (TIN) observed. The maximum scan angle or Field of View (FOV) is the angular extent measured in degrees of the view surveyed by the sensor. Absolute vertical accuracy achieved within land cover categories outside of bare open ground. Land cover categories specified in SVA Check Points shapefile. A polygon shapefile based on standard state indexes with an origin that aligns with the zero origin of the MGA Zone. The index defines how the LiDAR data is cut into tiles and supplied. The origin of the tile must be placed on a whole metre coordinate value of the south west corner of each tile. The tile name must be included as an attribute in the Tile Index file. A vector based representation of a surface made up of irregularly distributed nodes and lines with threedimensional coordinates (x, y, and z) that are arranged in a network of non-overlapping triangles pts/m 2 <= +/- 10cm. 95% confidence interval (1.96 x RMSE) RMSE z = [ (z data i z check i) 2 /n] 40 <= +/- 50cm. 95% confidence interval (1.96 x RMSE) 1km x 1km tiles based on MGA coordinates ar_data.pdf Page 60 of 78

61 9 Appendices 9.1 Appendix 1 How the Automated Checks are performed This Appendix provides details of the methods used by QA4LiDAR to perform the automated compliance and QA checks. QA4LiDAR is coded primarily in C# and uses ArcGIS (ArcObjects and geoprocessing tools etc.) and open source lastools Project scan in During project scan in, QA4LiDAR identifies the files that have been delivered by first identifying the file extensions and searching the file names using key term rules (see below example and Table 6). If the files cannot be identified from extension and file name, the search is extended progressively up the folder structure until the full directory path name is searched if necessary. This uses the prerequisite that tiled datasets must have their tile name within the file name (see Table 7 for rules used). As files are identified, they are added to the QA4LiDAR Project Database (.cqp). If a file cannot be identified, it is marked as unassigned. If duplicate datasets are found a dialog pops up to warn the user and instruct them to remove one of the duplicates, then try continuing the check or abort. File identification example File path 1 e.g.: C:\Dataset\LAS\C1\e123n4567.las Using the set of key term rules in the table below, as the file's extension is ".las" QA4LiDAR will use the LAS extension based rules to check the file path. First it will check for a match with classified/unclassified LAS by checking if the file path contains any of the classified/unclassified LAS rules. As the file path contains "C1", this is a match and the "Classified LAS" data type is assigned to the file. The rules are not case sensitive. As there is no datum reference in the path it will be deemed an ellipsoid LAS file as per the rule set. File path 2 e.g.: C:\Dataset\LAS\e123n4567.las No match would be found in the LAS rule set, as there is no key term for classified/unclassified, so no data type can be assigned to this LAS file. Table 6. Key term search rules (not case sensitive) LAS (.las) Data Type Key Term Rules File name or path contains term (not case sensitive) using any of the rules: "\xxx\", "-xxx", "_xxx", "-xxx_", "_xxx-", "xxx_" Classified: "cl, "c1", "c2", "c3", "c4" or "class" OR Unclassified: "unc" or "raw" (AND) *Type: "mkp AND *Tidal Datum: "lat", "mlw", "mhw" or "hat" OR ^Orthometric Datum: "ort", "ahd", "msl", "nzv", New Zealand local vertical datums; otp, akl, mot, gis, nap, tar, wel, nel, lyt, dun, dbl, blu or sti OR Ellipsoid: anything that is NOT one of the other datum options Page 61 of 78

62 ESRI Grid (.esrigrid fake extension given to ESRI Grid folder for QA4LiDAR) ESRI ASCII (.asc) ECW (.ecw) GeoTIFF (.tiff) or TIF (.tif) *Datasets found with these key terms are ignored in all but the first two check groups. ^Note that only a single orthometric datum is permitted within a project. Type: dem, bat (bathymetry), mix (mixed bathy/terrain) or * dsm OR *Intensity Imagery: int or dim AND Return Type: first, last or anything that is NOT one of the first or last return options is deemed all returns AND *Tidal Datum: "lat", "mlw", "mhw" or "hat" OR ^Orthometric Datum: "ort", "ahd", "msl", "nzv", New Zealand local vertical datums; otp, akl, mot, gis, nap, tar, wel, nel, lyt, dun, dbl, blu or sti (AND) *Mosaic: mosaic (if not a mosaic refer to Table 7 for tile rules) *Datasets found with these key terms are ignored in all but the first two check groups. ^Note that only a single orthometric datum is permitted within a project. Type: dem, bat (bathymetry), mix (mixed bathy/terrain) or * dsm OR *Intensity Imagery: int or dim AND Return Type: first, last or anything that is NOT one of the first or last return options is deemed all returns AND *Tidal Datum: "lat", "mlw", "mhw" or "hat" OR ^Orthometric Datum: "ort", "ahd", "msl", "nzv", New Zealand local vertical datums; otp, akl, mot, gis, nap, tar, wel, nel, lyt, dun, dbl, blu or sti (AND) *Mosaic: mosaic (if not a mosaic refer to Table 7 for tile rules) *Datasets found with these key terms are ignored in all but the first two check groups. ^Note that only a single orthometric datum is permitted within a project. *Aerial Imagery: rgb OR *Intensity Imagery: int or dim AND Return Type: first, last or anything that is NOT one of the first or last return options is deemed all returns (AND) *Mosaic: mosaic (if not a mosaic refer to Table 7 for tile rules) *Aerial Imagery: rgb OR *Intensity Imagery: int or dim AND Page 62 of 78

63 Shapefile (.shp) MapInfo TAB (.tab) ESRI Geodatabase (.gdb) Microsoft Excel (.xls) Microsoft Excel (.xlsx) Adobe PDF (.pdf) Microsoft Word (.doc) Microsoft Word (.docx) Extensible Markup Language (.xml) Comma Separated Values (.csv) Return Type: first, last or anything that is NOT one of the first or last return options is deemed all returns Contours: "contour" Flight Trajectory: "traject", flightline or flight_line Tile Index: tileindex, tilelayout, tile index, tile layout, tile_index, tile_layout, tile-index or tile-layout Survey Control: "control" Contours: "contour" Flight Trajectory: "traject", flightline or flight_line Tile Index: "tileindex", tilelayout, tile index, tile layout, tile_index, tile_layout, tile-index or tile-layout Contours: "contour" Flight Trajectory: "traject", flightline or flight_line Tile Index: "tileindex", tilelayout, tile index, tile layout, tile_index, tile_layout, tile-index or tile-layout Report: "report" Tidal Data: "tide" or "tidal" Report: "report" Tidal Data: "tide" or "tidal" Report: "report" Metadata: "metadata" Report: "report" Metadata: "metadata" Report: "report" Metadata: "metadata" Metadata: "metadata Survey Control: "control" Table 7. Tile name in file name search rules Data Type All tiled datasets i.e. LAS, DEM etc Tile Name Search Rules File name contains one of Rule Example e<eee>n<nnnn> e342n5820 <EEE>-<NNNN> _<EEE><NNNN> _ <EEE><NNNN> <EEE>_<NNNN> 342_5820 e<eeeeee>n<nnnnnnn> e342000n <EEEEEE>-<NNNNNNN> _<EEEEEE><NNNNNNN> _ <EEEEEE><NNNNNNN> <EEEEEE>_<NNNNNNN> _ Page 63 of 78

64 9.1.2 Delivery Completeness & Spatial File Corruption The checks in this group programmatically ensure that spatial data are not corrupt and that the delivery is complete i.e. all files requested have been delivered, the tile index coordinate origin is placed on a whole metre coordinate value that aligns with the zero origin of the MGA Zone, all tiles for tiled datasets are present (LAS are checked in this group and other tiled datasets and their mosaics in the next check group as missing LAS can halt the QA), if swath data was requested there is a swath LAS for every flight line in the shapefile, and if waveform LiDAR was requested checks there is WDP file for every waveform LAS. This also checks that the required elements on the Tender Form are selected on the Report Form. A failure in any one of these checks will halt QA4LiDAR and require action by the user before the automated checks proceed. This is to ensure time is not wasted running checks over data that is corrupt or incomplete. When checking all tiles for tiled datasets are present, only missing tiles in classified LAS datasets will halt the process as there may be legitimate reasons for missing DEM etc tiles e.g. in coastal areas, hence data types other than LAS are part of the second check group. QA4LiDAR uses the Tender Form to identify all the required deliverables. It then checks these requirements against the data identified in the scan in process. For tiled datasets it uses the tile names within the file names to match each file to a tile in the tile index. Each different spatial data file type has a different corruption test generally involving checking the header information matches the specification for the file and/or data within it. There are six parts to the LAS files corruption check; 1. File readability (core) a. The lasinfo tool is used to check readability and if a file is corrupt, the message will read Unable to open LAS file {file path}, it is corrupt. In this case the user should obtain new readable versions of these files before running other checks or they will crash. 2. Number of points is not zero (core) a. The lasinfo tool is used to check that the number of point records is not zero and if there are zero points, the message will read Error: {file path} contains 0 point records. The file is considered corrupt. In this case the user should obtain new readable versions of these files before running other checks or they will crash. 3. Number of points in flightline is less than 100 (core) a. The lasinfo tool is used to check if the number of point records in a flightline is less than 100 and if it is, the message will read Error: Flight line {PSID} contains less than 100 point records ({x} points). All files with this point source ID are considered corrupt.. In this case the user should confirm with the provider that the flightline is valid and if not have data re-delivered, as the flightline has very few points and will likely cause checks to crash. 4. Number of points is less than 100 (warning) a. The lasinfo tool is used to check if the number of point records in a file or for a PSID is less than 100 and if it is, the message will read Warning: {file path} contains less than 100 point records. The file is considered corrupt. In this case the user should consider removing such files from the delivery as they are suspected to cause issues running other checks. Page 64 of 78

65 5. File size (warning) a. This part simply provides a warnings to the user if the file size is greater than 2GB (ICSM specification requires LAS files <2GB). If this warning is given, the files can still be used in remaining checks. 6. Point source ID (warning) a. This part simply provides a warning to the user if the number of point source ID s per file exceeds QA4LiDARs expected maximum. The maximum expected number of point source ID s is 10 times the tile size in km, which is to ensure these numbers represent flight lines. If these warnings are given, the files can still be used in remaining checks however the files should be investigated to ensure PSID does represent flight lines. Shapefile corruption checks for the 4 essential files (shp, shx, dbf and prj) as well as a geometry check. The ASCII corruption check compares the number of rows and columns specified in the header, to the actual number in the data and if these match the data pass the corruption check. ESRI Grid, ECW and TIFF files are opened in ArcGIS and if the raster properties cell size X or Y value can be read successfully the files pass the corruption check. *Note. An ESRI ArcObjects bug has been found with ArcGIS version (no service pack) in which ecw files are deemed corrupt despite being able to be opened and viewed fine in ArcMap* To fix, please update ArcMap to File Naming, Shapefile Attributes & Horizontal Coordinate System The checks in this group programmatically ensure that all tiles for other tiled datasets (not LAS) and any mosaics are present, that the file naming conventions and file formats are as specified, that the attributes included in shapefiles are as specified, that the LAS headers are valid, that the definitions for horizontal coordinate system in the data match the Tender Form, that the tile size used matches the size requested on the Tender Form, and that the point source ID (PSID) for all points in each swath is valid if swath LAS were delivered. If one of the NEDF file naming conventions was specified in the Tender Form, file naming is checked against this specification (as defined by the relevant ICSM Template) by running files against a regular expression of the relevant convention i.e. using required characters, position of characters, order of characters etc. File naming convention example The Australian NEDF naming convention for classified LAS point clouds (as per the Aus ICSM Template) is: ProjectNameYYYY-CL-DAT_xxxyyyy_zz_wwww_hhhh.las The equivalent regular expression used is: \b[a-za-z0-9]+[0-9]{4}-c[0-9]-(ell AHD)_[0-9]{7}_[0-9]{2}_[0-9]{4}_[0-9]{4} Spatial files with required attributes as per the Tender Form are checked for these attribute fields. The lasvalidate tool is used to validate the LAS header information for each LAS file (this includes whether coordinate reference system information is present). The horizontal coordinate system definition in spatial files is compared to that required on the Tender Form. For files such as LAS that do not tend to have the horizontal coordinate system defined within the file or an associated projection file (.prj), the file name and path name are checked for key terms such as GDA, Page 65 of 78

66 MGA55 or just an MGA zone number such as 55 if the NEDF naming convention has been used. The horizontal coordinate system can also be visually checked at a later stage as part of the Extent Check. The tile size of the polygons in the tile index is calculated and checked against the requested size. The lasinfo tool is used to check that that there is only one valid (non 0) point source ID (PSID) for all points in each swath if swath LAS were delivered Comparison of Report Form to Tender Form This check compares the equivalent information on the Tender and Report Forms to ensure what has been reported by the provider matches the specification. The elements checked include; Report Form certified Form Required Elements (i.e. the number of requested datasets matches) Horizontal coordinate system Environmental conditions (yes/no) Absolute vertical accuracy Relative vertical accuracy Maximum scan angle Geoid model Classification level Vertical Reference system matches Minimum Bathymetry Coverage Minimum Bathymetry Soundings Classification Statistics This check uses the lasinfo tool. It first gets the list of required and must not have classes from the Tender Form. It then checks the classes contained in each ORT LAS file and determines whether each file passes or fails based on the list of required classes. Using the ArcGIS Point File Information tool, the Z minimum and Z maximum are recorded for each class in each LAS file. The project wide maximum and minimum for each class are then determined and reported. The number of points per class and the percentage of points per class are also calculated and reported. If ellipsoid LAS are present, the check is repeated for that dataset however the Pass/Fail criteria is based on whether the Point Count for each ellipsoid class matches the Point Count for the same class in the ORT LAS results. The lasinfo tool is also run on unclassified swath LAS, if they exist, to check the classes are all 0. If there is more than one type of LAS file in the project, the total point count between classified (orthometric and ellipsoid) and unclassified data is compared returning a PASS if the point counts match, or a FAIL if the point counts differ. Page 66 of 78

67 9.1.6 Accuracy of Survey Control This check ensures that the control points used by the LiDAR provider were collected to a minimum standard. This check can be run in the planning stages of a project to test the contractors plan for the control survey, if this is done the flatness and openness part is NOT run. If the check is run after data supply when there are LAS files in the project folder, the flatness and openness part IS run. The control should have their datum established independently of the LiDAR dataset. There are four Survey Control point types that can be input as shapefiles to QA4LiDAR; 1. Provider FVA Check Points (FVA CPs) o These are part of the Independent Check Point network and are used to assess the fundamental vertical accuracy of the survey. They must be gathered internal to the project area and are often collected in clusters on open, flat ground where there is a very high probability the sensor will have detected the ground surface, for easy comparison to LiDAR ground points. In QA4LiDAR they are used as part of the Survey Control Collection Method, Survey Control Density checks, the Survey Control Distribution check, as well as the Flatness, Openness & Absolute Vertical Accuracy checks. 2. Provider Ground Control (GC) o Along with Base Station data, GC make up what is termed the Control Network. GC are high accuracy points (e.g. state benchmarks) that are used by the provider to establish the datum in the survey area and adjust the LiDAR horizontally or vertically. They can be internal or external to the project and assess the variation of the reference surface and/or geoid model across the survey area. In QA4LiDAR they are used for the Survey Control Collection Method, Survey Control Density checks, the Survey Control Distribution check, as well as for Flatness, Openness & Absolute Vertical Accuracy checks. 3. Custom (Purchaser control) o This can be any other control (or shapefile elevation data of known accuracy) that the purchaser has access to. It will be used by QA4LiDAR for the Survey Control Density checks, the Survey Control Distribution check, and the Flatness, Openness & Absolute Vertical Accuracy checks. It should overlap with the survey area. 4. Provider SVA Check Points (SVA CPs) o These are part of the Independent Check Point network and are used to assess the supplemental vertical accuracy of the survey. They must be gathered internal to the project area and are located in a range of different terrain types. In QA4LiDAR they are only used as part of the supplemental Absolute Vertical Accuracy check. *Note. The Survey Control Accuracy and Absolute Vertical Accuracy Checks can be run with just FVA Check Points, with both FVA Check Points and Ground Control, with just custom (purchaser) control, or with all three of these control. If all three are supplied the checks will run using each type of control against each type of dataset. Before testing the Accuracy of Survey Control, if LAS data has already been collected and is present in the project folder, QA4LiDAR determines how open and flat the area surrounding each control point is, and hence its suitability for use in the network. This is done for all control points despite clustering, using ArcGIS methods. However, if LAS data are not present (i.e. you are checking the Page 67 of 78

68 accuracy of the survey control before acquisition), OPENNESS and FLATNESS fields are generated in the output control shapefiles and all populated with the value STRONG. These values of strong are not accurate ratings; they are simply for programming purposes so that all points are used in the Accuracy of Survey Control check. 1. The openness of the area around each control point internal to a convex hull minimum bounding geometry of the project extent, is rated. a) This is done by determining the percentage of non-ground classified (i.e. vegetation or building) LiDAR points (out of all LiDAR points) within a 10m radius from the control point. Ratings are based on the table below. b) Also, the classification of non-ground points are checked and if there are any class 5 (high vegetation) or class 6 (buildings) within the 10m radius, that control point cannot be rated better than Weak. c) If LiDAR points are only classified as ground (class 2) and unclassified (class 1) a rating will still be given however QA4LiDAR will not be able to take into account whether there are class 5 and 6 points. Table 8. Openness rating scheme Openness Rating % of non-ground points within 10m of control point Poor >50 Weak Average Good 5 15 Strong <5 2. The flatness of the area around each control point internal to a convex hull minimum bounding geometry of the project extent, is rated. a) The rating uses only the LiDAR points classified as ground within a certain radius of the control point. b) QA4LiDAR first counts the LAS points within a 1m radius of the control point, if there are less than 6 points the radius is increased by 25% and it searches again. This continues until at least 6 points are found. c) QA4LiDAR then determines the height differences of these ground points to the control point and averages these as absolute values. Flatness is rated for each control point according to the table below. Table 9. Flatness rating scheme Flatness Rating Average Height difference of ground points to control point (cm) Poor >8 Weak 6-8 Average 4-6 Good 24 Strong <2 Page 68 of 78

69 3. If the flatness or openness rating is unacceptable for a control point, that control point is not used in the Accuracy of Survey Control check. Whether a control point is unacceptable is up to the user. The user specifies a rating for each type of control in the Survey Control section of the Automated checks setup screen. Both flatness and openness for a control point must be rated equal to or better than the user s acceptability rating for the point to be used. The lower the acceptability rating selected by the user (i.e. closer to 'Poor'), the more control points that will be used for that type of control. These flatness and openness ratings are recorded in the output copies of the control data so the user can investigate the results. If a point falls outside the area of the LAS data it is given flatness and openness ratings of poor. There are then three aspects to this check; 1. Collection Method a) The QA4LiDAR Report Form is checked for the following information which is supplied to the user (the user requires some knowledge of control surveys to determine whether the methods and explanations provided denote adequate survey technique); i. Method used to collect GC ii. Explanation of GCP connection to orthometric datum iii. Method used to collect FVA CPs iv. Explanation of FVA CP connection to orthometric datum v. Method used to collect SVA CPs vi. Explanation of SVA CP connection to orthometric datum 2. Control Density a) QA4LiDAR checks whether an adequate number of points (GC and FVA CP) were collected for the survey area i.e. the density of the Survey Control. The survey area is determined from the original extent polygon supplied by the user. FVA CPs must be internal to the survey area and GCs may be internal or external. The density of the Survey Control is then rated in two modes. A Pass/Fail is given based on the following minimum points per square kilometre; vii km 2 : 5 FVA CPs + minimum 3 GC viii km 2 : 20 FVA CPs + minimum 5 GC ix. 400km 2 : 20 FVA CPs + 1 FVA CP for every 50km 2 over 400km 2 + minimum 5 GC b) In addition, a score as per the table below is provided for the density of points per square kilometre (e). For this score only FVA CPs and GCs internal to the survey area are counted. The density is given by; e = n a Table 10. Control density rating scheme where a = project area (km 2 ) Score n = number of internal FVA CPs & GCs Point Density 1 - Poor Weak Average Good Strong 0.02 Page 69 of 78

70 3. Distribution The distribution of FVA CPs and GCs internal to the survey area is rated. The ICSM Template (2010) defines the requirement as the distribution of FVA Check Points must be established to adequately cover the full extent of the survey area, and be representative of the project area landscape. This is determined employing ArcGIS methods for the following three part algorithm; a) Buffer: This is a ratio of the project area (blue area in diagrams) compared to the area covered by the FVA CPs and GCs (dots in diagrams) and determines the spread of points across the project area. i. The area per FVA CP/GC (b) is calculated using; b = a n where a = project area (m 2 ) n = number of FVA CPs/GCs E.g. a = m 2 n = 6 b = m 2 ii. The FVA CPs/GCs are spatially buffered with a radius (r) in m of; r = b π where b = area per FVA CP/GC calculated above E.g. r = 560m (i.e. radius of grey buffer circles) iii. The area of buffered points is spatially subtracted/erased from the original survey extent (resulting area in m 2 = g, orange area in diagram) and a numerical value for the spread (s) is determined using; s = a g a where a = project area (m 2 ) g = subtracted area (m 2 ) 0 s 1 E.g. g = m 2 s = 0.75 Page 70 of 78

71 The closer s ( Buffer value ) is to 1, the stronger the spread of Survey Control points. However, this does not account for all types of distributions and is combined with the following two scores (parts B and C) for a more accurate representation of distribution. b) Minimum Bounding Geomtery: i. This is the percentage of the project area outside the minimum bounding geometry of the FVA CPs/GCs which determines how well the FVA CPs/GCs cover the extremities of the project extent. ii. A minimum bounding geometry of the FVA CPs/GCs is created using a convex hull E.g. the yellow area iii. The minimum bounding geometry is erased from the project area (left with blue area above) iv. The area in m 2 of the remaining project area is calculated after the erase (blue area) v. This remaining area is divided by the original project area vi. The result is rounded down to one decimal place and subtracted from 1, to get a number between 0 and 1. This is the Min Bound value. c) Centroid: i. This is a ratio of the distance between the extent centroid and the FVA CP/GC mean centre, to the longest axis of the project area (in the X or Y direction to keep things simple) which determines whether the FVA CPs/GCs are weighted to one side of the project extent. ii. The project area centroid (blue point below) is found iii. The point mean centre (red point below) is found iv. The distance between the two points (blue and red) is measured in metres v. The longest axis distance of the project area is retrieved from the layer properties in metres i.e. top-bottom or right-left (arrow above) vi. The distance from centroid to centre is divided by the project area axis distance Page 71 of 78