STARTING WITH DRONES Data Collection and Remote Sensing with UAVs, etc. Dr. Bill Hazelton LS
What this Talk is About UAV-based data acquisition: What you need to get involved Processes in getting spatial information from the field to the client Airborne LiDAR and imagery (conventional aircraft): What s happening there and why drones are different A bit about the expansion of image-based data acquisition systems in terrestrial applications: Terrestrial LiDAR scanners, imaging total stations, digital cameras everywhere, Internet linking everything together What to think about when getting started, so no sales pitches!
About Bill Hazelton Surveyor (B.Surv. degree) for 39 years Licensed for 33 years Educator and Researcher for 30 years (PhD in 1992) Worked as a surveyor on three continents Surveying experience in: Cadastral Construction Business Management Geodetic Topographic Training and Education Photogrammetric Forensic Remote Sensing GIS/LIS Deformation Navigation
1. UAVs vs Regular Aircraft
UAVs / Drones A lot of hype about UAVs Unmanned Aerial Vehicles But UAVs are merely another platform to carry sensors: Satellites, aircraft, UAVs, regular instruments, instruments carried around The sensors are often scaled-down versions of conventional sensors: Cameras (visible and infra-red), video, scanners, LiDAR Geophysical sensors becoming available: Magnetic, electromagnetic, radar, gamma ray sensors So what s all the hype about? What do you need to know to get into this type of surveying?
UAV Advantages Low cost of acquisition and operation Able to be used in hazardous situation where aircraft and helicopters can t go Low-altitude surveys High-resolution and high-precision surveys Small, detailed and quick surveys where the cost of aircraft is not justified Some sensors can be tilted to allow oblique data collection Can collect imagery of the sides of building and steep-sloping terrain Can fly very close to objects to be measured Work beneath clouds in most cases
UAV Advantages Small-area surveys mean that ground control can be done quickly and cheaply: GPS RTK using a base station while the flights are being done Rapid deployment of equipment to the site Rapid changes of payload for multiple passes with different sensors Digital photogrammetric processing can produce a DEM in minutes Many systems can do flight planning and management with basic parameters entered by the operator quick and easy flight planning
UAV Disadvantages Small sensor systems Short range and endurance Limited payloads (this is improving rapidly) Multiple flights and recharges to accomplish project Affected by adverse weather, especially high winds Need to understand photogrammetry and remote sensing / image processing
Building the Project Does the client understand their own needs? If not, they cannot design the project, and so the RFP That may not stop them You may need to work with them to determine their actual needs Be connected to the decisionmaking process from start to finish
Planning This is something like traditional photogrammetric mapping: Look at specifications and link them to the capabilities of the equipment: This means you need to have tested your gear first Calibrate your equipment regularly, if possible with each job You can and should create and use a self-calibration range Many UAV mapping software packages will design the data collection flights to achieve the appropriate coverage: You need to have decided on the flying height to meet precision specifications before your software designs its flight pattern Need to know about planning for mapping, error analysis, decision support
Control Most UAV GNSS is not survey-grade, but it is getting better: Combined with an inertial measurement unit, it can get between 0.05 m and 0.3 m in location, 0.005 in roll and pitch, 0.015 in heading For many jobs, you will need ground control points However, you have to visit the site with the UAV anyway, so you can pre-mark some control points in high-visibility locations and determine their 3-D positions with RTK positioning, especially if the UAV can use RTK Because the UAV is close to the ground, your pre-marking can be fairly small and you can collect it afterwards and take it with you Design your control layout to minimize your error parameters
Calibration You have to calibrate the sensors you will fly on a job And this is not just cameras You may need to calibrate your sensors radiometrically as well as spatially Multi-spectral scanners need high-resolution color (frequency) calibration You can use colored boards and known ground conditions and check them with a handheld device Spatial calibration can be done with a test range More on calibration later: it s important! Need to know about error analysis
Flight Planning On-board software in a good survey UAV will deal with the details of getting the UAV to the right place and orientation for each image You still need to know how long the UAV can fly between recharges, and how much it can cover in that time This allows you to determine times, spare battery packs needed, and possibly the number of UAVs that need to be deployed to cover the job efficiently You will also need to figure out how many chargers to have, e.g., 30 minutes (max) flight duration, but 60 mins to recharge batteries fully, and two charging units to charge all 4 batteries (e.g., Riegl RiCopter) Knowledge of the terrain will allow you plan how long it will take to deploy, measure and retrieve control
Data Collection Do you have enough data storage? Plan for more data than you expect, to allow for repeat coverage A good laptop should hold everything in the field, but is there space? Back up everything, including to more permanent storage UAV-based LiDAR can get 350,000+ points per second Internal storage in the Riegl VUX-1UAV is a 240 Gb SSD (0.25 Tb) Add imagery and it gets quite large If you do a lot of this work, you may need a large data storage system and high-speed network in your office
Processing For UAV-collected data, processing depends upon the sensor Frame-based imagery (digital cameras) is processed using traditional analytical photogrammetric methods Scanned imagery is usually collected a line at a time (pushbroom scanner), and is processed one line at a time, if high precision is required LiDAR is processed almost a point at a time, but also warped along flight lines to match ground control and cross-lines Video is usually not processed for spatial uses, but frames may be Geophysical data needs to be processed and managed according to its needs Spatial control for imagery is usually sufficient for geophysical data
Processing Traditional photogrammetry, as well as more recent Shape/Structure from Motion (SfM) methods, match clusters of pixels in different images and deduce the location of the object in the object space (real world) Traditional photogrammetry does it by analytical geometry SfM does it slightly differently under the hood The results are similar There is free photogrammetric software available: https://en.wikipedia.org/wiki/comparison_of_photogrammetry_software. (103 entries) There are also many proprietary systems
Processing For processing, you will need to understand how the photogrammetric restitution process works While software does a lot of the work for you, you still need to understand the processes involved Begin with registering the images with the camera calibration data: This allows the correct distortion parameters to be included in the processing from the start The second stage is identifying common points between overlapping images Software can do some of this, but human eyes and brains can deal with difficult cases
Processing With a large number of common points identified between images, the images are resampled to convert them to epipolar images This is the equivalent of relative orientation and allows relative tilts between images to be determined and corrected for Epipolar images allow rapid searching of the images to find common points, and so create the 3-D representation (model) of the object space Noting the control points in the images allows the resulting 3-D representation to be brought into the same reference system as the control points The final 3-D basic representation of the object space can now be created
Processing From this basic 3-D representation, a range of products can be derived: For example, contour plots, surface plots, 3-D renderings The imagery can be draped over the surface, creating an orthophoto Fly-thoughs and various video presentations can be created Note that these products are fairly traditional and largely visual These are the more usual mapping products However, you have this representation in a digital format Digital data lends itself to further processing and analysis Need to understand processing data
Analysis You rarely need to map an area just to make a map There is usually a need to learn more and to support a decision This requires analysis and usually mixing with other data A GIS is one approach for this, CAD is another, depending on the problem The critical points in the analysis, together with the critical decision points, shape the data needs, and so need to be considered at the earliest planning stages A major advantage of UAVs is their low cost, allowing a job to be re-flown if the data are not helpful to the decision process Need to understand decision processes and how data errors affect them
Visualization Mapping is one thing, but you need to present the results to the client Traditionally, we produced maps and plots, both paper and digital Clients may now want outputs visualized in different ways Consider the growth of Virtual Reality (VR) capabilities as a means of exploring multi-dimensional data ($20 headset, software, your smartphone) Some photogrammetric software can provide enhanced 3-D visualization Can you provide a 3-D representation with live connections to a database about the objects appearing in the visualization? Can you provide field visualization on mobile devices?
Follow-through How well did the client understand the problem? This dictates how well the solution meets their needs Many clients are thinking of the technology of 20-30 years ago and ask for far less than can be delivered for the same price Many of the client s ideas may not answer their problem Consider developing expertise in problem analysis This can allow you to develop very tight specifications for the project, because you know what are the critical decision points, and so exactly what to measure, how well to measure it, and where and when it needs to be measured to support the decision that has to be made
Rinse. Repeat. For a complex problem, you may need to map the problem first, so you know what the critical parameters are UAVs allow quick mapping in reconnaissance mode, which can be done fairly cheaply, e.g., fly transects to sample the region This allows you to decide where the denser data collection needs to be done, and which areas can be covered thinly Expensive with conventional aircraft, cheap with UAVs UAVs allow this iterative approach, which may be novel for a client, but can make certain projects far more efficient and effective
What You Need to Know for Projects The theoretical and technical photogrammetry concepts, for frame imagery Equivalent foundation for other types of data collected Error analysis and propagation Decision support concepts and processes Mission planning for each data sensor and process Analysis potential for your collected data Visualization and representation of your data and results Connections between these and the other things that may done to your data
2. CALIBRATION
Calibration Cameras and other imaging devices can produce results that are good to ridiculously fine tolerances Cameras can also throw substantial errors into the derived data through various forms of distortion, without images appearing distorted Calibration is essential to determine that the measurements are realistic Stability of calibration is also critical Avoid cameras with moving parts in the lens system Metric and non-metric cameras what s the difference? While self-calibration is critical for cameras and other sensors, radiometric calibration can also be critical for certain sensors
Radiometric Calibration Test if the frequency (color) recorded by the sensor matches the source Important for multi-spectral scanners, but not particularly for cameras Tends not to drift over time, but a correction table may need to be provided Test by comparing the signal from the sensor agrees with what is tested by an independent device There are handheld devices to obtain that data Other sensors may need other types of calibration, e.g., geophysical sensors
Geometric Calibration Geometric capabilities of the sensor, especially cameras, need to be calibrated For a frame camera, an array of known points that cover the field of view reasonably well is required These points should be in a location that represents where the UAV will operate, and be able to be imaged from a regular flight Could be close to the office for easy flight testing Control points should be visible, flat and easily imaged from a wide range of angles Marking in a way that allows the control points to be seen in images is important
Geometric Calibration Calibration of scanners is more difficult, but a test range is a possibility Pushbroom scanners can be calibrated over a test range For multi-spectral scanners, alignment of the different bands is important, but changes little over time Radiometric calibration is more important Calibration of LiDAR requires a test range of known points with targets, but for UAV LiDAR a series of well-placed small reflectors provide very bright returns LiDAR calibration also needs to deal with the GNSS/IMU, as this also provides part of the location
AIRBORNE LIDAR AND IMAGING DEVELOPMENTS
Meanwhile, back in the air While there has been a rush to miniaturize sensors for UAV deployment, the conventional aerial side of the house has been hard at work The miniaturization has allowed more to be packed into existing packages While there are some fairly basic pushbroom scanners for UAVs, there are very advanced system for aircraft The Leica ADS100 is designed largely to replace aerial cameras It is not that it provides greater precision, but it cuts a large amount of time from post-flight processing LiDAR is also becoming more advanced, being able to be flow higher and faster, provide multiple returns and use far weaker return signals
Aerial Systems The decreased weight of scanners and LiDAR has allowed the addition of more sensors A recent Reigl sensor has two LiDAR heads and two digital cameras built into a single camera port, providing greater coverage More systems can be flown on the one aircraft, allowing denser data collection More varied systems can be run at once, e.g., geophysical sensors Still not enough miniaturization for SAR and IfSAR in UAVs IfSAR still requires considerable power and space in conventional aircraft
TERRESTRIAL SYSTEMS
Terrestrial Developments For terrestrial systems, there are progressively more imaging systems being incorporated Total stations are becoming support platforms for a range of other sensors GPS/INS are evolving to support imaging systems, and are in the earliest stages in smartphones The key development is moving more and more data collection to dumb 3-D points, rather than more traditional individual point collection We add the meaning and intelligence in post-processing
THANK YOU!