MAPPING CAPABILITY OF A LOW-COST AERIAL DATA ACQUISITION PLATFORM FIRST RESULTS

Transcription

1 MAPPING CAPABILITY OF A LOW-COST AERIAL DATA ACQUISITION PLATFORM FIRST RESULTS Hannes Raggam, Roland Wack, Karlheinz Gutjahr Joanneum Research, Institute of Digital Image Processing Wastiangasse 6, A-8010 Graz, Austria hannes.raggam@joanneum.at; Facsimile: Commission VI, WG VI/4 KEY WORDS: low-cost platform, GPS, IMU, aerial data acquisition, building extraction, surface mapping ABSTRACT: Based on a first prototype, an improved low-cost aerial platform was developed at the Institute of Digital Image Processing in order to be used for immediate acquisition and processing of image data wherever necessary. Typically, such requirements arise in case of natural hazards, where the amount of environmental or infrastructural damage has to be rapidly assessed and quantified. In general, the platform comprises a camera as well as GPS and IMU instruments in order to record the position and orientation of the camera. Using a helicopter, first campaigns have been flown with this platform over various urban and sub-urban test sites in Vienna and Graz in order to map buildings, or more generally - to create digital surface models. Ground control points were measured and used for these test sites to perform aero-triangulation via block adjustment, and to estimate and validate the potential accuracy with respect to 3D mapping. In this paper, pilot mapping applications are presented, which refer to interactive 3D building extraction using photogrammetric workstation software on the one hand, and widely automated surface model generation using image matching procedures applied to multiple image data on the other hand. 1. INTRODUCTION Due to an increasing amount of natural hazards in the past years, reliable and up to date image information becomes more and more important, e.g. to monitor environmental changes, to protect and monitor sensitive infrastructure, or for an effective disaster management and decision making processes. As an example, geo-referenced high resolution aerial imagery and 3D surface models can serve as an appropriate information source, in particular if the processing of the acquired data can be carried out in near real time. The Institute of Digital Image Processing has set up an airborne platform for a low cost data acquisition and fast product generation. The platform allows rapid and versatile data acquisition whenever and wherever needed. Applications being typically envisaged are mapping of disaster events like flooding, land slides, storm damage and the like. Thereby, the objects and areas of interest can be imaged by as many overflights and as many repetitive images as deem to be reasonable. Rapid mapping of such events is an obvious objective, requiring automated execution of individual data processing steps. In order to launch first data acquisition campaigns, the platform has been mounted on a helicopter. Images were captured over selected urban and sub-urban areas in or in the vicinity of the cities of Vienna and Graz. Forward and backward over-flights were made, each of them leading to about 5 coverages of the objects on ground. Hence, a highly redundant image material was achieved, which can be utilized in manifold alternatives for 2d and 3d mapping tasks. In the following the aerial data acquisition platform is described in more detail. For 2 selected test sites, the procedures and the results of mapping experiments devoted to building and surface mapping are presented. On the one hand, this platform setup was used to provide image data within a project that required a 3D building reconstruction for a wave propagation simulation. On the other hand, surface mapping procedures utilizing, which utilize multiple image coverage as provided by a high overlap of sequential images, were applied for 3D surface mapping for a selected test site. The benefit and the algorithmic issues of these procedures are presented and discussed. 2. AERIAL DATA ACQUISITION PLATFORM Prior to a decision on the specific components of the platform the basic requirements have to be outlined. The final version of the aerial data acquisition platform should provide multiple overlapping images for a 3D surface reconstruction and the feasibility of a quasi true-ortho image and mosaic generation. For disaster monitoring near real time image processing is required, implying geo-referencing without using ground control points for optimization and validation purposes. Further requirements refer to low cost, easy operating and high flexibility. This flexibility requires a low weight platform that can be mounted on an aircraft (helicopter or airplane) or can simply be operated hand-held also from a non expert. The camera resolution should be as high as possible in order to assure a reasonable trade-off between resolution on ground and area being covered 2.1 First platform prototype The first platform prototype was realized by a high resolution digital camera (12 mega pixels), which was further connected to a L1/L2 GPS phase receiver and operated from board of a helicopter (see also Table 1). A related data acquisition experiment was made in the context of landslide mapping. After a period of intensive rainfall at the end of August 2005 in the area of Gasen (located in the north eastern part of the Austrian

2 province Styria) several landslides caused severe damages on houses and infrastructure. Above all one person was killed by a collapsing house. For this area, aerial images were captured with an overlap of at least 70 % to enable the generation of a 3D surface model. The images had to be taken in an oblique (off-nadir) viewing direction with respect to the area of interest. The average height above ground was 400 meters, which yielded to an image resolution of about 15 cm. In total more than 200 images were necessary to cover the entire area. The procedures and results achieved for this mapping experiment were presented in Raggam et al. (2006). 2.2 Current platform conception Based on the experiences of the mapped landslides near Gasen the platform setup was changed. The capacity of the 12 mega pixels camera is low in case that a larger area is to be covered by images of high pixel resolution. Higher data acquisition efficiency hence requires a higher camera capacity. Therefore the camera was replaced by a Hasselblad H2D with 39 mega pixel. In addition a low cost IMU (inertial measuring unit) was mounted on the camera, thus providing approximate values of the camera s exterior orientation (see Table 1). Platform configuration Camera system Camera resolution GPS system First prototype consumer camera Current conception high end consumer camera Future conception high end consumer camera 12 Mp 39 Mp 39 Mp L1/L2 Phase receiver L1/L1 Phase receiver L1/L2 Phase receiver + EGNOS IMU system - X-Sense Novatel IMU accuracy - low high Stabilization - - yes Image processing post processing post processing near real time sequential images was in the order of about 70 % (image example see Figure 2). 3.2 Ground control For the determination of the exterior orientation a couple of ground control points (GCPs) and verification points were measured for each of the test sites by means of a GPS survey. For the verification of the GPS positioning accuracy a second survey of GCPs was carried out for the shopping city Seiersberg test site one week after the first measuring campaign. The average horizontal and vertical deviation of these points measured in two epochs was 2.5 cm and 4 cm, respectively. For the test site Mariatrost no GPS measurements were made. Instead, GCPs were measured using available ortho-photo maps and an existing elevation (terrain) model, implicating a reduced GCP accuracy in an estimated order of a few decimetres in planimetry and about 1 meter in height at the best. Figure 1. Aerial image of test site Seiersberg Table 1. Airborne data acquisition platform concepts (EGNOS: European Geostationary Navigation Overlay Service) 3.1 Image data 3. DATA ACQUISITION First data acquisition campaigns based on the current platform conception were driven by a project of the European Space Agency (ESA), which was devoted to the characterization of the satellite to indoor channel at S band. Therefore, 3D building models together with material attributes were required. These data should be derived by interactive stereo data processing. Image data were acquired for six selected test sites in Austria, representing prominent buildings like the airport, the millennium tower or the FFG building in Vienna, or the airport, the shopping city Seiersberg or a prefabricated house park in Graz. Each of the test sites has an extension of about 500 by 500 meters and was covered with images at 80 % overlap and a ground resolution of 8 cm from board of a helicopter. In total, some 25 images were captured for each site (image example see Figure 1). In addition to the six sites the sub-urban test site Mariatrost, which is located near Graz, was captured. Here, the overlap of Figure 2. Aerial image of test site Mariatrost 3.3 Mapping Objectives According to the objectives of the ESA project on the one hand, and in order to test the developments made at the institute with respect to the utilization of multi-image coverage for surface mapping on the other hand, the following tasks and experiments were applied to these data sets: Interactive mapping of selected buildings. Therefore, the commercial Leica Photogrammetriy Suite (LPS) was

3 used. In this paper, results achieved for the Seiersberg test site are presented. Automated surface mapping utilizing multiple image data sets. Therefore, recently developed approaches included in the Remote sensing Software package Graz (RSG) were used. In this paper, first results achieved for the Mariatrost test site are presented. RSG is a development of the Institute of Digital Image Processing with a strong focus on 3D mapping using optical as well as SAR image data. For these first mapping tasks, still un-calibrated image data have been used due to unavailability of reliable calibration parameters, e.g. to cope for radial image distortions. As will be shown, this leads to distinct mapping discontinuities, which become particularly obvious if not only single stereo pairs are treated. 4. BUILDING EXTRACTION For the characterization of the satellite to indoor channel at S band 3D building models together with material attributes were required. To generate the CAD building models 3D points needed to be measured in the stereo images. Finally the models were setup within the commercial Autocad software. In the following, results achieved for the test site Seiersberg are presented. 4.1 Block Adjustment / Seiersberg For this test site, a core of 18 images was collected into an image block and imported into LPS. Using 17 GCPs which were measured appropriately in the images and numerous automatically detected tie-points a block adjustment was carried out to determine the exterior orientation of the images. From the control points, 3D setup/mapping accuracy in planimetry and height could be estimated as follows (bias/mean and standard deviation values given): Mean: m E = 0.00[m] m N = -0.02[m] m H = 0.05[m] Std.Dev.: s E = 0.07[m] s N = 0.10[m] s H = 0.26[m] These accuracy estimates can be considered to be sufficient in order to meet the accuracy requirements for the intended building mapping D Modelling In the following the images were imported into Leica s Stereo Analyst. The shapes of all relevant building features as well as the terrain were stereoscopically measured in individual stereo models. The 3D point cloud which was captured in that way was transferred into Autcad in order to finally create a surface/building model (see Figure 3). Road tracks and parking areas are shown in gray, the terrain in green, a building activity area in light brown, and building blocks appear in different colours depending on the materials of the façades and the roofs. 5. SURFACE MAPPING APPROACHES UTILIZING MULTIPLE STEREOSCOPIC COVERAGE 5.1 Background As mentioned above, the low-cost aerial data acquisition platform can be operated in a way that highly overlapping sequential images are achieved for an object or an area of interest, either in a single or in multiple over-flights. This leads to multiple stereoscopic coverage, which further on may be utilized for surface reconstruction purposes with distinct benefit in comparison to standard stereo pair utilization. At the Institute, first implementations and experiments have been devoted to image triplets (Raggam, 2005), while a set of 5 UltracamD images over the forested area mentioned above was used by Ofner et al. (2006) in order to carry out a multi-stereo image surface mapping experiment. The benefit of multiple stereoscopic coverages, i.e. more than 2 overlapping images for a target point on ground, may be discussed by means of Figure 4. Here, sequential image pairs may be matched and 5 lines of sight be achieved from 4 matching results in the ideal case. On the one hand, the utilization of 5 lines of sight instead of only 2 makes the point intersection procedure to determine the ground coordinates of a target point more robust and leads to a distinct increase of surface model quality. On the other hand, the occlusion effects caused by trees or buildings are reduced in comparison to standard stereo pairs. Forest gaps may for instance be imaged from 2 neighbouring images (e.g. images 2 and 3 in Figure 4), while they might be invisible for a standard stereo disposition (e.g. images 1 and 5). Besides, neighbouring images in general show a higher similarity, leading to a superior matching performance in comparison to the more diverging standard stereo images. Figure 3. 3D model of the shopping city Seiersberg Figure 4. Multi-image mapping scenario

4 5.2 Image registration In order to facilitate image matching, a (coarse) registration of the respective image pair is reasonable. The surface mapping procedures which are at present implemented in the RSG software can deal with the following options: Utilizing stereo images: One image is coarsely registered with respect to the other by means of a linear (and hence invertible) point transformation, which may be derived from a set of tie-points. These points either may be measured manually, or may be determined automatically using the imaging equations and a coarse elevation (terrain) model. The images may be physically registered in advance, or the registration may be implicitly considered during image matching. Utilizing stereo ortho images: The images are orthorectified using a coarse terrain model. It is an obvious benefit that image matching can then make use of well registered input image pairs in order to determine the remaining ortho image disparities, which are due to features like vegetation and buildings, which are not represented in the terrain model. A comparative analysis of standard stereo mapping versus ortho-image based stereo mapping is given by Gutjahr et al. (2005) for glacier mapping using Eros and Ikonos image pairs. These 2 general approaches to extract 3D information from overlapping stereo images were expanded with respect to the utilization of more than one image pair, i.e. multiple matching results originating from sequential stereo pairs as indicated in Figure 4 are to be used simultaneously in order to reconstruct the terrain surface. The approaches to utilize multiple stereo ortho image disparity maps and stereo image disparity maps are described in the following sections. point in image 1 (P1) points to the corresponding location P2 with respect to ortho image Then, all matching results where ortho image 2 was used as reference image are investigated, e.g. those resulting from matching ortho image 2 and 3, 2 and 4 etc. A disparity vector is interpolated at location P2 of these disparity maps, again pointing to the corresponding location of the search image (e.g. image 3) and defining a location P3. 3. The procedure is continued with remaining disparity maps. This procedure to collect all corresponding matching results is supposed to deliver in a first instance multiple ground coordinates (P1, P2, etc.) of a certain feature on ground. These may be more or less severely different due to the displacement of this feature in the individual ortho images. These ground locations are back-projected into the individual images to achieve multiple image coordinates, defining a couple of lines of sight. 5.3 Image Matching The dominating proposition for image matching is to allow matching of individual image pairs as most convenient and promising. It is suggestive to match image pairs sequentially, i.e. the first with the second, resulting in a disparity map DM-12 according to Figure 5, the second with the third (DM-23) etc. according to the sequence of image acquisition. This strategy assures that image pairs with highest similarity are used in the matching procedure. Nevertheless, also matching results achieved from e.g. the first with the third image (DM-13) etc. should be applicable to the follow-on surface reconstruction procedure. 5.4 Disparity Map Tracking A key task to utilize multiple disparity maps resulting from a matching strategy as indicated above is to collect corresponding matching results, i.e. those referring to the same target point. The multiple matching results lead to multiple lines of sight for this target point. Spatial intersection of these lines of sight yields the target point coordinates on ground Tracking Stereo Ortho Image Disparity Maps: The collection of corresponding matching results over a sequence of disparity maps is done as follows: 1. The first disparity map (resulting from matching the first and the second ortho image) being included defines 2 lines of sight to those targets which were matched successfully. Each disparity vector determined for a target Figure 5. Scheme of multiple image overlaps Tracking Stereo Image Disparity Maps: While ortho image based disparity maps can be tracked via the ground coordinates being defined by the pixel and the disparity as such, auxiliary means have to be used in the approach which was implemented in order to track stereo image disparity maps. The following facts and preparatory steps are considered: The coarse (linear) transformation between 2 images (see section 5.2) is used for tracking, as it also gives a link between individual (specifically consecutive) disparity maps. E.g., the transformation between images 1 and 2 also defines the relationship between disparity map DM-12 and disparity map DM-23. Consequently, appropriate (linear) transformations can be established between arbitrary disparity maps by merging/chaining individual transformations. E.g., a transformation between images 1 and 4 becomes feasible by merging the transformation between images 1 and 2, 2 and 3, and 3 and 4. For tracking, a reference disparity map is selected (e.g. the first in the sequence, like DM-12), and

5 transformations with respect to all other disparity maps being involved are established. Only disparity maps, where the reference images is either the same as for the reference disparity map, or where the reference image corresponds to the search image of another disparity map can be utilized. E.g matching result using images 1 and 2 (DM-12) cannot be linked immediately with the matching result using images 3 and 4 (DM-34), while it can be linked with the matching results using images 2 and 3 (DM-23), or 2 and 4 (DM-24). Disparity map tracking is done using the above mentioned transformations in order to collect all matching results corresponding to the same target from the individual disparity maps. In its general workflow, this procedure is equivalent to the one applied for stereo ortho image disparity maps (see section 5.4.1). 5.5 Spatial Point Intersection Multiple matching results, which are collected by disparity map tracking, result in multiple lines of sight, which are then subject to spatial point intersection. Here, a least squares approach is applied in order to determine the location of the corresponding feature on the ground. It is to be noted that projection lines resulting from a single matching result of neighbouring images would represent a weak geometric disposition to determine the corresponding point on the ground due to their small intersection angle. However, the collection of several corresponding matching results in general yields a comparably wide-spread bundle of multiple projection lines, assuring numeric and geometric stability for the determination of ground coordinates. The spatial intersection procedure further includes several mechanisms to identify and reject erroneous or highly uncertain matching results. Such unreliable matching results for instance use to occur in or close to areas which are occluded in one of the images to be matched. Back-matching thresholds and/or thresholds of the point residuals resulting from the least squares intersection procedure are applicable to reject doubtful matching results. Based on the results of spatial point intersection a raster surface model can finally be generated. 6. MULTI-IMAGE SURFACE MAPPING RESULTS The approaches to extract surface models from multiply overlapping - optionally ortho-rectified - stereo pairs as described in section 5 were applied to the test sites Seiersberg and Mariatrost. In the following, results achieved for the suburban test site Mariatrost are presented. 6.1 Block adjustment For this test site, 17 images were collected into an image block. Using a set of 31 GCPs and 39 additional tie-points, which have been measured for the images of this test site, a block adjustment was applied yielding the following results with respect to 3D setup/mapping accuracy: Mean: m E = -0.09[m] m N = -0.01[m] m H = -0.13[m] Std.Dev.: s E = 0.22[m] s N = 0.21[m] s H = 0.84[m] These results are distinctly worse than those achieved for e.g. the Seiersberg test site. However, they correspond well with the accuracy potential/limits induced by the used reference data D Surface Mapping For a first test of the surface mapping approaches outlined in section 5, 4 sequential images were selected for this test site (see Figure 6). Matching was applied to the 3 sequential (ortho) image pairs and surface models generated from the matching results. The matching disparities as well as the surface models are shown in Figure 7 for the stereo image and for the stereo ortho image based approach. The disparities are shown in yellowish colour, while blue colour indicates areas of poor matching reliability, like typically forest or building borders, or at the image borders. The stereo image disparity maps as well as the stereo ortho image disparity maps were then tracked in order to collect corresponding matching results, to perform spatial point intersection and to finally generate a surface model. The surface model resulting for both approaches are shown in Figure 7. In general, the 2 surface mapping results are well comparable. Many of the unreliable matching results can be removed in the over-determined least squares point intersection procedure by introducing appropriate thresholds for the backmatching distance and or point intersection residuals. Major problem areas or features, however, still can be observed. Besides, there are obvious discontinuities in the surface models, which exactly correspond to the borders of individual disparity maps, or stereo coverage. The observable height discontinuities are in the order of a few meters. Although a proof could not yet be made, it can be assumed with high confidence that these artefacts are due to the missing image calibration. Image calibration tests, which have been carried out meanwhile, have shown that (radial) image distortions of some 20 pixels in length have to be accounted for at the image borders. This undoubtedly can lead to height errors in the above mentioned order of magnitude. 7. CONCLUSIONS Based on the first results, which have been achieved by interactive as well as automated 3D mapping procedures, the following major conclusions can be made with respect to the mapping capabilities of the low-cost data acquisition platform: 3D mapping using image data acquired by this aerial platform can be done with sufficiently high quality and accuracy The multi-image based surface reconstruction approaches have an obvious potential to circumvent problems inherent to standard stereo mapping. Provided that images are acquired with a high overlap, these approaches use to facilitate image matching, to reduce unreliable matching results and to reject such results in case they occur. Image calibration is an essential prerequisite to assure high quality as possible. As a consequence, instantaneous activities are devoted to the latter issue in order to either compensate for the image distortions by generating calibrated input image data, which are not affected by distortion effects,

6 or to include and consider the respective distortion parameters resulting from the calibration activities within the underlying imaging equations. The surface mapping experiments then shall be reproduced and continued including other test sites, in order to comprehensively evaluate the 3D mapping potential of the low-cost data acquisition platform in its present conception. 8. REFERENCES Gutjahr K., Raggam H., and Franke M., Herleitung digitaler Oberflächenmodelle aus hochauflösenden Stereo- Bilddaten im alpinen Gelände. Österreichische Zeitschrift für Vermessung und Geoinformation (VGI), 93. Jahrgang, Heft 2/2005, pp Raggam H., Surface Mapping using Image Triplets - Case Studies and Benefit Assessment in Comparison to Stereo Image Processing. In Photogrammetric Engineering and Remote Sensing, Vol. 72, Number 5, May 2006, pp Hirschmugl M., Ofner M., Schardt M., and Raggam H., Single tree detection in very high resolution remote sensing data. Remote Sensing of Environment Special Issue ForestSat, in press. Raggam H., Wack R., and Gutjahr K., Assessment of the Impact of Floods using Image Data acquired from a Helicopter. 26th Earsel Symposium, New Developments and Challenges in Remote Sensing, Warsaw, May 29th June 1st, ACKNOWLEDGEMENT Image calibration tests were made by V. Kaufmann and R. Ladstätter from the University of Technology, Graz. Stereo mapping on the LPS was done by F. Schöggl, H.C. Haslauer, C. Schreiner and C. Sommer, and supervised by V. Kaufmann. These contributions are thankfully acknowledged. Figure 6. Example of 4 sequential images of test site Mariatrost (flight direction left to right). Stereo image disparity maps Surface model derived from sequential stereo image disparity maps Stereo ortho image disparity maps Surface model derived from sequential stereo ortho image disparity maps Figure 7. Surface models derived from multiple stereo image pairs (left) and multiple stereo ortho image pairs (right).