Automated Orientation of Aerial Images

Transcription

1 Automated Orientation of Aerial Images Joachim Höhle Aalborg University Denmark Abstract Methods for automated orientation of aerial images are presented. They are based on the use of templates, which are derived from existing databases, and area-based matching. The characteristics of available database information and the accuracy requirements for map compilation and orthoimage production are discussed on the example of Denmark. Details on the developed methods for interior and exterior orientation are described. Practical examples like the measurement of réseau images, updating of topographic databases and renewal of orthoimages are used to prove the feasibility of the developed methods. 1 Introduction The production of maps and orthoimages requires the orientation data of the aerial images. This unproductive part of the mapping process should be done at low costs and reliably. Generations of researchers have worked hard to improve this process. The use of digital images gave new possibilities improving the orientation process and reducing the expensive fieldwork. Furthermore, maps and orthoimages are nowadays stored in databases together with many additional data. Their updating at short intervals of time is a current task in many countries. This type of work and the generation of thematic maps have to be carried out by non-photogrammetrists as well. The orientation process should therefore be simple and automatic. The search for new methods for that task was the goal of an OEEPE project as well as of continuing research at Aalborg University. It is the aim of this paper to summarize the results and to evaluate the potential of the developed methods. 2 The updating of topographic databases In contrast to a complete new mapping the updating of maps has to be done in small and distributed areas, namely where changes in the landscape took place. The additions should fit optimally into the existing data, and the relative accuracy is therefore more important than the absolute accuracy. The tasks involved are detection of changes, orientation of images and production of orthoimages or updating of the topographic databases. The procedures and products in the production differ somewhat from country to country and from organisation to organisation. In the following the example of Denmark is used in order to give an idea, which kinds of products and maintenance tasks exist. The products are digital maps and topographic databases, orthoimages and digital terrain models and a variety of other databases with geodata.

2 2.1 Digital maps and topographic databases in Denmark Topographic objects of cities are mapped after specifications for technical maps (T3/TK3/TK99). All objects (features) are tagged with a code, and all the individual points of an object are recorded with YXZ-coordinates. The smallest objects to be mapped are drain gratings, sewer manholes and masts. Well-defined point objects are mapped with a standard deviation of σ p = 0.10 m (or σ Y,X =0.07 m) in planimetry and σ h =0.15 m in height or better. Wide-angle photography has to be taken at a scale of 1: Technical maps of rural areas are derived from photography 1: , and they have to meet other specifications (T2/TK2/TK99). A nationwide topographic database (TOP10DK) was compiled from photography 1: to 1: in stereo workstations. The database contains about 50 different objects, for example the centreline of roads and buildings with a minimum size of 25 m 2. The accuracy of welldefined objects is quoted to be σ p = σ h =1.0 m. Private mapping companies using analytical stereo plotters or stereo workstations compiled all of the mentioned vector data. The updating of the TOP10 DK is planned at intervals of 5 years. The private Danish photogrammetric companies store (natural) control points for future use. 2.2 Orthoimages The private company Kampsax/Geoplan has produced orthoimages for the whole country since This so-called Danish Digital Orthoimage (DDO95) was produced from colour photography in the scale 1: with a pixel size of 0.8 m. It was renewed already in These orthoimages (DDO99) have a pixel size of 0.4 m on the ground. A new production is planned for Currently colour orthoimages of large cities (DDOtown) are produced with a pixel size of 0.1 m using normal-angle photography in the scale of 1: Digital terrain models The digital terrain model used for the orthoimage production of the DDO95 and the DD99 was derived by image correlation and by additional editing in stereo workstations. The accuracy of this digital terrain model (DTM) is quoted to be σ = 1-2 m. Terrain data used for the production of DDOtown are derived from airborne laser scanning. The distance between the height points is 1 m, and the accuracy of a single point is σ = 0.15 m. Another countrywide terrain model is computed by the National Survey and Cadastre (KMS) from 5 m contours. The accuracy of this DTM is quoted to be σ = 1-2 m; the distance between heights is 50 m. 2.4 Other geodata databases A variety of other countrywide databases containing geodata is produced, for example the Danish Address and Road Database (DAV), the Housing and Dwelling Register (BBR), the Digital Surface Model (DSM), the Digital Elevation Model (DEM), the Clutter Database. The DEM contains the elevations of buildings, forests and other objects above the terrain, the Clutter Database contains a classification of the landscape each 5 m (towns) or 20 m (other areas) with 8 or 4 classes, respectively.

3 3 The necessary accuracy for the orientation of aerial images The orientation of images has to be done with such accuracy that the requirements of the mapping tasks can be met. Map compilation and orthoimages have different requirements. The use of digital photogrammetry offers new possibilities of improving the accuracy of the mentioned products. 3.1 Accuracy of digital photogrammetry Practical mapping still uses film-based cameras and scanners for the conversion to digital images. The geometric quality of the scanner and the selected size of the pixel influence the accuracy of measurements. Systematic errors of the scanner are eliminated by a transformation using calibrated parameters. The deformations of the image are corrected by measurement of the fiducial marks and a transformation to the calibrated coordinates of the fiducial marks. Small random errors remain. At the exterior orientation of images several control points have to be measured. Well-defined control points can be measured with a third of the pixel size or better. If the photographs are scanned with 14 µm x 14 µm large pixels, the measuring accuracy is about 5 µm. The accuracy of measured image coordinates is estimated to be σ x', y' = 6 µm. Such accuracy was for example found in a recent OEEPE test using signalised control and check points and conventional bundle adjustment (Heipke, C. et al., 2001). Taking such accuracy of the image coordinates into account, the three rotation angles of a wide-angle camera (c=152 mm) can be reconstructed with an accuracy of σ ϕ = σ ω = 2.5 mgon and σ κ = 3.3 mgon, respectively. If natural control points are used (as it is the practice in Denmark), the accuracy of the image coordinates will be less, because these points are not so well defined. 3.2 Map compilation Digital maps are compiled in digital stereo workstations. In todays practice the results of an aerotriangulation (parameters of the exterior orientation for two overlapping images) are transferred to the stereo workstation. The stereo models should then be free of vertical parallaxes. Such parallaxes limit the measuring accuracy and cause model deformations. The aerotriangulation is calculated with up to 21 additional parameters (in order to compensate for various deformations); they are not usable by the software of the stereo workstation. This causes vertical parallaxes as well as errors in planimetry and height. In this investigation natural control points and coordinates from a topographic database will be used for the orientation of the images. A definition error has to be taken into account. The overall planimetric accuracy for a control point is then: σ xy = σ xy( sig) + σ xy( def ) The overall height accuracy is found in the same way. The height accuracy of signalised points is determined by: σ z( sig) = 2 σ xy( sig) / n = σ xy( sig) n base to height ratio (n=0.61 at wide-angle photography and 60% forward overlap).

4 The definition of topographic objects can be taken from table 1: Object σ xy(def ) [cm] σ xy(def ) [cm] average σ z (def ) [cm] σ z (def ) [cm] average Manhole covers House & fence corners Field corners Tab.1: Definition accuracy for different objects [taken from (Waldhäusl, 1980)]. From Fig.1 it can be seen that the obtainable accuracy for the exterior orientation differs with the image scale (or flying height) and the used map object. The use of manhole covers (σ xy (def) average = 5 cm) and an image scale of 1:5000, for example, results in an accuracy of σ x',y' =12µm. Fig. 1: Accuracy of image coordinates (σ ) as function of the image scale and the definition accuracy of various map objects: manhole covers (mhc), house & fence corners (hfc), field corners (fc). The dashed lines are height accuracies in %o of the flying height (h). It can also be seen from Fig.1 that the use of natural points has a considerable influence on the achievable accuracy of the orientation. Only the manhole covers will enable an acceptable accuracy for largescale mapping. The small-scale mapping can use objects, which are not well defined. 3.3 Orthoimage production The accuracy of the orthoimage depends on the accuracy of the orientation data and on the accuracy of the digital terrain model.

5 2 2 2 ortho Ori DTM (1) σ = σ + σ In a DTM all heights are on the ground. If a DTM is used in the production of an orthoimage, all objects above the terrain, for example buildings, are displaced in the orthoimage. Such objects should therefore be avoided as control points. True orthoimages can be generated if a DSM is used. Such orthoimages will be produced of urban areas only, and only a few examples are known today. Countrywide orthoimages are based on DTMs. The accuracy of an orthoimage is influenced by the accuracy of the heights (σ z ), the type of the terrain and the distance between the points. 2 2 σ DTM = σ Z + σ 2 Terrain The influence of the distance between points ( X) on the accuracy of the model for a given terrain type can be estimated by means of formulae derived in (Jacobi 1977): 2 Terrain = ( X ) σ (Danish moraine landscape) If the countrywide DTM of the Danish National Survey and Cadastre is applied (σ z =1-2 m, X=50m), the total error of the DTM at Danish moraine landscape will remain 1-2 m. Height errors in the DTM have a varying effect on the orthoimage. The size of the planimetric error depends on the location in the orthoimage (compare Fig. 2). If, for example, the DTM has a height error of σ DTM =1m, a planimetric error of 0.70 m (RMSE value) can be expected. If the countrywide orthoimage should have an accuracy of 1m, errors caused by an erroneous orientation of the image should not exceed σ Ori =0.7 m according to (1). If the orientation of the images were error-free, the planimetric error in the orthoimage would still be 0.7 m. When optimal accuracy of the image coordinates (σ x', y' = 6µm) is achieved, the planimetric error on the ground would be σ p = 21 cm (assuming an image scale of 1: ). Such a high accuracy is not necessary for orthoimage production, because the planimetric errors due to errors of the DTM are 3.3 times bigger. Consequently, the accuracy requirements for the orientation data can be reduced by the same factor, i.e. σ ϕ, ω = 8 mgon and σ κ = 11 mgon in the considered case. Fig. 2: Isolines of planimetric errors in an orthoimage due to a height error of σ=1m in the DTM. Values are in meter.

6 4 Automated methods for the orientation of images 4.1 General remarks The use of image processing techniques enables automation of the orientation of images. At the interior orientation well-defined fiducial marks (or réseau marks) have to be measured. Various authors have solved the automation of this task and elaborated software programs are included in several digital stereo workstations today. At the exterior orientation of images the image coordinates of signalised control points or of natural points have to be measured. Their position in a reference system has to be known. If suitable databases exist, the necessary control information can be extracted and expensive field surveys can be avoided. The exterior orientation by means of control information extracted from databases was a research project of the OEEPE. Various authors contributed with new solutions and applied it to the same test material. The applied methods and their results were published in (Höhle 1999). In the meantime new investigations have been carried out. In the following chapter the methods developed at Aalborg University will be described. 4.2 Methods for automated orientation Interior orientation The automatic measurement of well-defined crosses has been investigated in (Höhle 1997). The applied method derives a template interactively from a sample cross and uses it as the measuring mark for other crosses. The position of the crosses is determined with subpixel accuracy and high speed. A transformation of the measured pixel coordinates onto the calibrated values of the crosses will reveal corrections, which then can be applied to other imaged points. Fig. 3 depicts a part of the developed program. Fig. 3: Interactive derivation of a template for the automated measurement of crosses. The upper image is a sample cross which was extracted from a réseau camera image. The lower image is an interactively derived template. The keys on the right side are used to adjust a cross for geometry and the cross as well as its background for radiometry. The derived template is then used as the measuring mark for the automatic measurement of the crosses of the réseau camera image.

7 4.2.2 Exterior orientation of single images The exterior orientation of a single aerial image can be determined by means of several control points. Their image coordinates and ground coordinates are the observations in the collinearity equations, and the exterior orientation parameters of an aerial image are computed in a least squares adjustment. Image patches of an aerial image and an orthoimage replace the control points. Correspondence between pixels of both images can be found by cross correlation. A template is shifted pixel by pixel within a search area, and a correlation coefficient is computed at each position. Correspondence is at the pixel with the highest correlation coefficient. The method is depicted in Fig. 4. column Fig. 4: Matching between patches of an aerial image, and the existing orthoimage is used to determine the exterior orientation of the aerial image. aerial image row orthoimage The images are of different dates. It is therefore necessary to select image patches which contain time-invariant objects. A variety of databases can support the selection of usable patches. For example road crossings are pretty stable over the years and contain sufficient structure and contrast. The spatial coordinates of road crossings can automatically be extracted from a topographic database, and image patches can be extracted from the orthoimage and the aerial image. Approximations of the exterior orientation are required in order to have a limited search area. Mismatches may occur due to the changes in the landscape. These mismatches have to be detected and eliminated automatically. Such blunder detection is possible when many pairs of image patches are at disposal. More details about the method can be found in (Höhle, J. & Potucková, M., 2001a/b) Exterior orientation of stereopairs The exterior orientation of two images can be found in a similar way as for one image. A template of a map object is projected onto both images using approximate values of the exterior orientation. Because the aim is large-scale mapping, the method has to achieve subpixel accuracy and to cope with coarse approximations for the exterior orientation. In the method developed by B. Pedersen the approximations known from flight planning are improved stepwise using image pyramids and object pyramids. Parts of roads are used in lower resolution levels of the images and manhole covers or drain gratings in the level with the highest resolution. At this zero level of the image pyramid a subpixel is derived by means of the correlation coefficients around the position, where the correlation coefficient has it maximum. A polynomial function is used for modelling a surface. The coefficients of the polynomial function are estimated by least-squares adjustment, and the highest point of the calculated surface is found by simple formulae. This automatically derived position is the best fit between the two image patches, and the coordinates are determined with subpixel values and a measure of accuracy

8 (standard deviation). In order to detect blunders in the automatic measurements the maximum correlation coefficient and the standard deviation of the coordinates are compared with selectable thresholds. If these thresholds are surpassed, the measurement will not be used in the bundle adjustment. Furthermore, robust adjustment is applied at the bundle adjustment in order to detect and eliminate remaining gross errors. A large number of observations is, however, necessary. Redundancy and hierarchy are the main principles in this approach (Pedersen 1999). The used objects (parts of roads, manhole covers and drain gratings) are automatically extracted from the database and converted from vector representation into raster representation. Assumptions for the grey value and the contrast to the surroundings have to be made so that the templates correspond to the majority of the imaged objects. Also the size of the point-like objects (e.g. the radius of the manhole covers) has to be estimated. 5 Results of various tests The developed methods were applied in practical applications. Some of the results are summarized in the following. 5.1 Measurement of réseau camera images (interior orientation) The used image was taken with a Rollei 6006 réseau camera, which has a format of 55 mm x 55 mm where 121 crosses are engraved on a glass plate. The distance between the crosses is 5 mm. The exact coordinates of the crosses can be taken from the producer's calibration certificate. The imaged object is a concrete plate whose deformations have to be determined by photogrammetry. The object differs in brightness, and other objects sometimes disturb the imaged crosses. The image was scanned in a KODAK scanner, and the data have been stored on a Photo-CD at five different resolutions. A template was derived interactively from a sample cross and contained 11 x 11 pixels. The search area was selected with 15 x 15 pixels. This means that the initial positioning of the search area has to be within ± 2.5 pixels with regard to the exact position. This was achieved by measuring 3-4 crosses manually in advance. The derived transformation parameters are used to control the positioning of the search areas. During the automatic measurement process the current search area is superimposed over the image and the grey values are read. Intermediate and final results are displayed and stored. In this way problems with positioning and/or in the correlation were observable. An affine transformation between the measured pixel coordinates and the calibrated coordinates of the crosses revealed that there were deficiencies in the used scanner with regard to orthogonality of the two axes and their affinity. The latter means that the pixel was not squared. The mean square error (RMSE) of the residuals was 0.24 pixels. The measurement is faster and more accurate than with manual operation. The automatic measurement can be incorrect under certain circumstances. It must, therefore, be a goal for good software development to detect blunders. A threshold for the maximum correlation coefficient and the estimated accuracy of the measured image coordinates can be introduced as a first measure. The graphic display of residuals of an affine transformation will also reveal blunders in the automatic measurement. An interactive

9 or semiautomatic procedure therefore seemed to be the best approach for an accurate measurement of targets in digital images. 5.2 Exterior orientation of single images applied to automated orthoimage production By means of test data of the Danish National Survey and Cadastre, an orthoimage from 1995 (0.625 m pixel), the topographic database TOP10DK (σ=1 m), a DTM (σ=1-1.5 m, X=25m) and a recent aerial image (1999, 1:25 000, 25 µm pixel), a new orthoimage was generated. The exterior orientation data were derived by means of the described method. 81 image patches of 31x31 pixels were extracted from the old orthoimage. Fig. 5 depicts a section of the map together with the used patches. Fig. 5: Road crossings extracted from the topographic database and the old orthoimage. The orthoimage patches are used as templates in the search for similarity within the (new) aerial image. Patches of 61 x 61 pixels were then extracted from the aerial image. Z-coordinates of the road crossings and approximations for the exterior orientation were used in the computation. A threshold for the maximum correlation coefficient was set to T1>0.7 and the threshold for the image area covered with patches to T2>75 %, which eliminated 54 % of the measurements. The remaining 37 image patches were used for the matching, and the coordinates of the aerial image were replaced accordingly. The exterior orientation of the aerial image was calculated by means of a newly developed bundle adjustment program (Höhle 2001). Residuals after the first and the following iterations were used to derive weights for the observed image coordinates. The weight function was changed after the third iteration. The final solution of the exterior

10 orientation was obtained when the changes in the square sum of the residuals were less than 2% or when 10 iterations were carried out. The new orthoimage was then derived by means of the commercial software package "BaseRectifier" from Z/I Imaging. In the computations the following parameters were chosen: 25 m distance between anchor points, bi-cubic interpolation for the determination of the grey values and m pixel size in the generated orthoimage. In order to check the achieved accuracy, coordinates of 25 road crossings were measured in both orthoimages. The coordinates were compared and a root mean square error of R = 0.9 m derived. This error was computed from the differences (dx and dy) after n number of road crossings R= ( dx + dy ) 2n The achieved accuracy is better than the accuracy of the vector map (1.0 m), which was used for the extraction of the road crossings. A closer investigation revealed a systematic shift of 0.7 m in the X-coordinate and 0.1 m in the Y-coordinate between the two orthoimages. If this systematic error could be avoided the error would be R=0.6 m only. If the used roads are superimposed onto the old as well as on the new orthoimage, deviations between the orthoimage and the centreline of the roads could not be noticed. 5.3 Exterior orientation for stereopairs applied for updating of large scale vector maps Existing vector map data of the Danish TK3 standard (σ p =10 cm, σ z =15 cm) were used for the derivation of orientation data for two new images (c=153 mm, 1:5000, 15 µm pixels). The available approximations for the orientation data were up to 71 m and 3.5 gon off from the final results. Road parts, manhole covers and drain gratings were extracted from the given map data, projected onto both images and converted into raster representation. The squared patches then served as the template, which was moved pixel by pixel within a search window of 81 x 81 pixels. Matched positions were found at each level of the pyramid by means of the LSM method. The pixel coordinates were accepted as final, when the maximum correlation coefficient was larger than 0.75 and the standard deviation of the subpixel values less than 0.1 pixel. The image coordinates and YXZ coordinates of the corresponding points were used as observations in a robust bundle adjustment program which calculated the final orientation parameters. 38 % of the 60 image measurements were removed by thresholding. The calculated standard deviations of the unknowns of the exterior orientation were: σ Y = 0.21 m, σ X = 0.13 m, σ Z = 0.10 m, σ ω = 21 mgon, σ ϕ = 23 mgon and σ κ = 21 mgon. The values are the average of two images. With the derived orientation 25 drain gratings were mapped. Their coordinates were compared with the reference values, and the following RMSE values were obtained: Y = 0.08 m, X = 0.10 m, Z = 0.13 m. These are the results of B. Pedersen with the OEEPE test material. Three other participants of the OEEPE test obtained similar results with other methods. A student project at Aalborg University confirmed that with the use of drain gratings and similar data a sufficient accuracy for the orientation parameters can be obtained and that the applied methods are practicable (Falk, J. and Nielsen, N.J.D., 1999).

11 6 Conclusion In order to make the orientation of images faster and cheaper the measurement of image coordinates has to be automated. Blunder detection is very important for such processes. Thresholds for the correlation coefficient and the standard deviation of the subpixel value as well as robust adjustment are the tools for detecting and eliminating blunders and gross errors. The combined use of various existing database information (e.g. orthoimages, vector data and DTM data) can prevent expensive fieldwork. Their accuracy and consistency has to be checked in advance. The automatic extraction of usable map objects and the generation of templates can easily be accomplished. The methods for single images and models are also applicable for blocks of images. The images are then first tied together by aerotriangulation procedures. Approximations for the exterior orientation are necessary; they can be received from navigation data. Larger search windows and a combination of object and image pyramids can also solve this task with the disadvantage of longer calculation times. The updating of topographic databases and orthoimages, using such orientation procedures, enables high neighbouring accuracy, and new and old data fit optimally. References Falk, J., Nielsen, N. J. Dalum, 1999, Automatisk måling af paspunkter med henblik på ajourføring af TK3, Final project at Aalborg University, Denmark Heipke, C., Jacobsen, K., Wegmann,H., 2001, The OEEPE Test on Integrated Sensor Orientation Analysis of Results, proceedings of the OEEPE-workshop on Integrated Sensor Orientation, 2001 Höhle, J., 1997, The Automatic Measurement of Targets, Photogrammetrie, Fernerkundung, Geoinformation 1/97, pp Höhle, J., 1999, Automatic Orientation of Aerial Images on Database Information, OEEPE Official Publication N o 36, pp , ISSN Höhle, J., 2001, Automated Georeferencing of Aerial Images, proceedings of ScanGIS'2001, Ås, Norway Höhle, J. & Potucková, M., 2001a, Steps to Automated Orthoimage Production, proceedings of the International Symposium on "Geodetic, Photogrammetric and Satellite Technologies Development and Integrated Application", Sofia, Bulgaria Höhle, J. & Potucková, M., 2001b, Towards the full automatic production of orthoimages, Photogrammetrie, Fernerkundung, Geoinformation 6/2001, S Waldhäusl, P., 1980, proceedings of the XIVth ISPRS Congress, Commission V, Hamburg, Germany