THE IMAGE ANALYST AND GEOSPATIAL ANALYST WORKSTATION

Transcription

1 END-TO-END PHOTOGRAMETRY FOR NON-PROFESSIONAL PHOTOGRAMMETRISTS Kurt J. de Venecia, Product Manager BAE Systems 8400 E. Prentice Ave., Suite 1500, Greenwood Village, CO 80111, USA Rick Racine, Product Manager BAE Systems Sunset Hills Rd., Reston, VA 20190, USA A. Stewart Walker, Marketing Director BAE Systems Technology Pl., San Diego, CA 92127, USA ABSTRACT BAE Systems is a leader in geospatial image processing using a wide range of image sources. SOCET SET and VITec ELT are two examples of commercial products BAE Systems has developed for photogrammetry and image analysis. These products cover two ends of the imagery exploitation spectrum SOCET SET for high-end production photogrammetry and VITec ELT for easy-to-use image viewing, mensuration, annotation, and analysis. Automation in the photogrammetric process is a key component that allows the image analyst to produce sophisticated photogrammetric products such as triangulation, terrain models, and orthomosaics. In depth analysis of triangulation results is not practical for an image analyst, who wants a large area coverage mosaic as an output product. This paper reviews the automation of the photogrammetric process for the image analyst who may use disparate image data sets from multiple sensor types and multi-temporal acquisition to create orthomosaics. The automated process includes: 1. Automatic tie point measurement using area-based and feature-based matching on multiple sensor types with temporal variations. 2. Triangulation of the results from automatic tie point measurement on the multiple sensors simultaneously. 3. Automatic terrain extraction using bare-earth filtering. 4. Large area coverage orthomosaic with automatic seamlines and image balancing. The automated process creates a large area coverage mosaic using disparate imagery with varied a priori accuracies. THE IMAGE ANALYST AND GEOSPATIAL ANALYST WORKSTATION The Image Analyst (IA) is someone who typically works with georeferenced source imagery. The end products of the IA process are usually time critical. Therefore, the amount of time spent with individual image acquisitions may be short. A critical component to utilizing a georeferenced image is the sensor model. The sensor model is used in the transformation of object space coordinates to image space. The IA is typically not concerned about the underlying sensor model, but wishes to achieve a high level of accuracy when deriving end products from the source imagery. Typical IA jobs include screening imagery and creating end products such as annotated chipped images with map grids. Other products might include large area mosaics, perhaps contain map grids. IA products from BAE Systems include the legacy VITec ELT software and new SOCET GXP software.

2 The Geospatial Analyst (GA) is someone who also works with georeferenced source imagery, but unlike the IA, the GA may work on an image over a period of days. The GA products may include triangulation of input source imagery to update sensor model parameters in order to achieve a more accurate ground to image transformation. Other products include digital elevation models, feature data, orthophoto mosaics, etc. GA Workstations from BAE Systems include the legacy SOCET SET software and the SOCET GXP integrated IA/GA software solution. The development of an IA/GA workstation necessitates high accuracies in processing georeferenced source imagery for both communities of end-users without two different user interfaces and complex manual intervention including analysis of results. The IA may wish to triangulate imagery, but may not have the training or background to perform such a task. The GA may also wish to triangulate imagery, but may like more access to the error propagation model in order to refine the sensor model adjustment. Both IA and GA production flow lines may start with triangulation and proceed through terrain extraction and on to orthophoto mosaics. There are many other product flow lines for both the IA and GA, but this paper will concentrate on this example. The IA workflow may necessitate deriving a large area coverage orthomosaic process without the sometimes tedious terrain editing and QA step or cut sheets for orthomosaic products such as Digital Ortho Quads or Control Image Base. The editing of terrain and cut sheets for orthomosaics is typically the responsibility of the GA. In either workflow, the automation of the photogrammetric process is a key component for product generation in a timely manor. Automated Triangulation Computation of ground locations from imagery requires sensor modeling that convert coordinates from ground space to image space, or vice versa. The sensor model is used by nearly every photogrammetric application on the IA/GA workstation. A detailed review of the sensor modeling employed by SOCET SET is described in Olander and Walker, A sensor model in SOCET SET/GXP consists of four components: Support data input/output Ground-to-Image function (or, alternatively, image-to-ground function) Triangulation Error Propagation Model Support data includes information such as sensor location, velocity, orientation angles, focal length, time of acquisition, and camera calibration data. Support data is often supplied in a header file attached to the raw imagery or a separate metadata file associated with the image. The sensor model must be written to include a direct ground-to-image or an image-to-ground transformation. If the ground-to-image function is provided, then the image-to-ground function can be computed numerically using an iterative algorithm. Conversely, if the image-to-ground is provided, then the ground-to-image can be computed iteratively. If speed of execution is a concern, both functions can be written explicitly. Triangulation interface functions include: Identification of the sensor model parameters that are adjustable Determination of default accuracies for each adjustable parameter Inputs to an automated triangulation application are the sensor model as well as tie and optional control points. Full automation does not have the benefit of using control points measured by an operator. Therefore, well behaved tie point matching algorithms along with apriori knowledge of the sensor accuracy components are used to refine the individual sensor parameters of multi-source and/or multi-temporal georeferenced imagery. Two algorithms are employed to match imagery in SOCET SET/GXP. The two algorithms are referred to as an area matcher and an edge matcher (feature matcher). The processing employed by the matching algorithms is controlled by the triangulation manager. The triangulation manager utilizes strategy files that contain default values, which can be fine tuned by experienced operators. The area matcher is an enhanced version of the Automatic Point Measurement application used in SOCET SET for nearly a decade (de Venecia, K., et al, 1996). Improvements to the processing include a new Local Bundle Adjustment (LBA) as well as robust local grid matching. The basic algorithm uses normalized cross correlation as well as a derivation of the Förstner interest operator. Since correlation is looking for like details on imagery, it is typically very successful on like imagery. For example electro-optical (EO) imagery like SPOT that does not contain a large amount of temporal variation. In addition, the area matcher has proven to be successful on matching like imagery from different sensors, such as EO imagery from SPOT, IRS1C, QuickBird, and IKONOS.

3 A complementary algorithm to the area matcher is a new edge matching algorithm, which has its legacy in a program called HART High Accuracy Registration and Targeting (Lofy, 2004). The algorithm is typically employed when there are significant temporal variations in the imagery, or the imagery was acquired with different types of sensors. The basic algorithm matches linear features between these disparate image types. Since the edge matcher breaks an image up into linear components, it can successfully match imagery from different sensors such as EO and synthetic aperture radar (SAR). The Universal Triangulation (UTri) employed by SOCET SET and many other programs at BAE Systems, allows for simultaneous adjustment of multi-sensor imagery. Proper weighting of sensor parameters allows the UTri to adjust poorly defined sensors (such as SPOT 1-4, or IRS1C) to fit higher accuracy sensors (such as QuickBird and IKONOS). The UTri is capable of eliminating measurement errors, while at the same time maintaining adequate point distribution to support good geometric modeling. The outputs from UTri have an error propagation model that allows an IA or GA to determine the accuracy of data collected from the resultant georeferenced imagery. The accuracy for individual points or features is typically reported as a CE and LE, Circular and Linear Errors, respectively. Automatic Terrain Extraction The extraction of terrain data from stereo imagery is a product for both the IA and GA workflow. The terrain data is used directly for processing such as precise point mensuration, perspective scene generation, visibility analysis, navigation. It is also used in the end-to-end photogrammetry process for products such as orthophoto mosaics. The Adaptive Automatic Terrain Extraction algorithm used by SOCET SET (Zhang and Miller, 1997) process requires input source imagery that has been triangulated resulting in the elimination of y-parallax errors. The source imagery must have sufficient convergence to allow for accurate determination of elevation data at specified ground locations. Optional inputs for the algorithm include existing elevation data such as DTED. Existing elevation data is used to seed the AATE process allowing the existing elevation data to be updated in the stereo overlap region. Additionally, the existing elevation data can be used to fill-in areas where non-stereo imagery exists, which will be used in the orthomosaic processing. Recent improvements to the AATE algorithm include the addition of a bare-earth filtering algorithm, based on continuity of elevation data when compared with neighboring elevations along with constraints on slope tolerances. The algorithm is loosely based on Zhang et al, In order to process imagery through the automated end-to-end workflow, future enhancements planned for March 2005 are in work to allow the AATE to match on multiple overlapping images for a single elevation data point. Currently, the AATE uses a single pair based on image convergence for the image correlation process. Large Area Mosaic Processing of the input imagery from automated triangulation through automatic terrain extraction allows for these products to be input into the mosaic application. The large area coverage orthomosaic process has default values for items such as the output Ground Sample Distance, boundary, mosaic method, interpolation method, image balancing, etc. A simple subset of the mosaic parameters can be overridden by the IA operator. By default the mosaic application determines the order of mosaicked imagery to be predicated by the most nadir option. This option works well for vertical aerial imagery, but may not be the best method for multi-temporal imagery with large differences in spatial resolution. In this case, the operator may wish to override the defaults and select a mosaic method such as best spatial resolution, or date of acquisition. The large area coverage mosaic product must include metadata allowing for the product to be used within the IA/GA workstation such as SOCET GXP, or input into another application or imagery database. Therefore, the output image format from the automated end-to-end production of an orthomosaic is either NITF or GeoTIFF. SOCET GXP can be used to add margins, map grids, scale bars, legends, and annotations for complete product generation.

4 PRACTICAL RESULTS OF THE END-TO-END PHOTOGRAMMETRY WORKFLOW FOR THE NON-PHOTOGRAMMETRIST Workflow Manager The Visual Coverage Tool (VCT): The selection of inputs for the end-to-end photogrammetry workflow is controlled by the Visual Coverage Tool application. The VCT is tightly integrated with SOCET SET/GXP. It provides a graphical display of images and terrain superimposed on CADRG Maps. The operator is required to select input directories for imagery and terrain. After the selection of the directories, the image and terrain footprints are displayed. The selection of the data to be processed is controlled by the operator, who outlines a bounding rectangle around the area to be processed. Figure 1 illustrates the simple user interface provided by VCT. Figure 1 also illustrates the simple checkbox options for Triangulate Images and Orthomosaic Images. Figure 1. Visual Coverage Tool graphical display used for workflow management of the end-to-end photogrammetry processing for the non-photogrammetrist. Inputs: The test data used in the end-to-end automated workflow is outlined in table 1. The ten images used were all acquired from satellites and had spatial resolutions ranging from approximately 0.7 m for QuickBird to 12.5 m for RADARSAT. Review of the imagery in the table shows considerable variations in the image content due to time. There is significant snow on the SPOT 4 image and excessive clouds on the IRS1C image. It is also important to note that multi-band and multi-byte imagery was used as input. Limitations on the use of this data in conjunction with single byte and single band imagery will be outlined in the mosaic processing section. The initial testing of the automatic triangulation process with the raw input imagery proved to be unsuccessful due to the large amount of displacement between the initial sensors parameters provided with each of the input source images. The maximum displacement was almost 5 km between the high accuracy QuickBird and IKONOS images and the lower accuracy IRS1C, SPOT 4, and RADARSAT images. In order to provide reasonable inputs to

5 the process, a single tie point was measured between the 10 images to allow for the adjustment of sensor positions or offset terms of the individual models. Additional inputs included two DTED level 1 files and four DTED level 2 files.

6 Imagery IKONOS (4) Pan Sharpened 3 band 11 bit/band Stereo Epipolar RPC Sensor Model Ground Sample Dist: 1.0 m Acquisition (yyyymmdd): Size (lines x samples): 11080x x x x8420 Thumbnail QuickBird (2) Basic Panchromatic 11 bit Quaternion Sensor Model Ground Sample Distance: 0.7 m Acquisition yyyymmdd: Size (lines x samples): 28592x x27552 SPOT 4 (1) Level 1A 8 bit/pixel Physical Pushbroom Sensor Model Ground Sample Distance: 10.7 m Acquisition yyyymmdd: Size (lines x samples): 6000x6000 IRS-1C (1) Standard Left PAN Sub-frame 8 bit/pixel Physical Pushbroom Sensor Model Ground Sample Distance: 5.9 m Acquisition yyyymmdd: Size (lines x samples): 14798x4096 RADARSAT (2) Path Image Plus Processing 11 bit/pixel Radar Spotlight Mode Sensor Model Ground Sample Distance: 12.5 m Acquisition yyyymmdd: Size (lines x samples): 7803x x9150 Table 1. Ten disparate satellite images of the Denver, Colorado area used for end-to-end photogrammetric processing for the non-photogrammetrist.

7 Triangulation: The triangulation processing for the ten input satellite images used both the area matcher and edge matcher to determine tie points. The footprints of the input imagery as well as the resultant tie points are displayed in figure 2. Two examples of tie points measured by the area and edge matcher algorithms are displayed in figures 3 and 4. Figure 2. Image footprints for the ten input satellite images as well as resultant tie points from the area and edge matcher. The image on the left provides an overview of the image block, while the image on the right shows a zoomed in area allowing review of the multi-image matches (2-7 images for each tie point). Figure 3. Example of a three ray tie point measured between RADARSAT, IRS1C, and SPOT 4 imagery. All images are displayed at the same approximate scale and rotation.

8 Figure 4. Example of a tie point measured on seven different images. The top 4 images are two image acquisition events for the two IKONOS pan-sharpened stereo images. The bottom three images from left to right are two QuickBird images and a single SPOT 4 image. All images are displayed at the same approximate scale and rotation. The results from the area and edge matcher were processed by UTri. The default accuracies for all sensor model parameters were used as were the adjustable parameters selected. The triangulation manager application controlling the matchers as well as UTri processed the data in five passes. The final pass contained no measurement blunders and resulted in a final RMS pixels for the 63 tie points measured. Terrain Extraction: The user can select terrain for the orthophoto mosaic to be derived from input imagery or existing DTED. Although not fully implemented at the time of this paper, the orthophoto mosaic will also include an option to use terrain data from a combination of existing DTED and AATE processed. This capability already exists in the SOCET SET AATE, which allows for the input of a seed DTM. The seed DTM can be an existing DTED. The March 2005 planned enhancements to AATE will allow the application to match all convergent stereo pairs and update the DTED with the best match. At the time of publication, the author had to process the terrain manually by selecting the IKONOS stereo images for AATE processing and existing DTED cells as the seed terrain information. After selecting the inputs, the process continued unaided and produced terrain data used to create the final mosaic product. Orthophoto Mosaic: Options for creation of the final large area mosaic include the use of existing DTED or terrain data derived from imagery as described above. Other options include the selection of the output spatial resolution (GSD). The user interface for selecting orthophoto mosaic options is shown in figure 5.

9 Figure 5. Current options included in the creation of the large area coverage mosaic output product. The resultant orthophoto mosaic was created using 5 input images processed through triangulation as described above. The mosaic process does not allow for mixing of imagery with varying bit depth or bands. Therefore all images for the mosaic process were down-sampled to 8 bits/pixel. Enhancements planned for the end-to-end processing include allowing the mosaic processor to work with inputs of varying bit depth and bands. The resultant 5 m orthophoto mosaic is shown in figure 6. Figure 6. Resultant orthophoto mosaic using two QuickBird images, a single SPOT 4 image, and two RADARSAT images. The zoomed panels show the seamlines between the RADARSAT and SPOT 4 images on the left, and QuickBird and SPOT 4 images on the right.

10 CONCLUSION An end-to-end photogrammetry solution for the non-photogrammetrist is available now. SOCET GXP and SOCET SET both provide this solution with the Visual Coverage Tool. Very few limitations exist and plans exist to fix the limiting factors as described in the terrain and mosaicking sections of the practical results section. The biggest hurdle has been conquered by employing two tie point matching algorithms along with a robust Universal Triangulation solution. Imagery Analysts and other non-photogrammetrists can rely on SOCET GXP with VCT to provide an automated photogrammetric solution for a variety of orthomosaic production needs. ACKNOWLEDGEMENTS The authors would like to thank the data providers SPOT, Space Imaging, and DigitalGlobe. We would also like to thank the VCT and VCT Batch development team of Johnathan Simmons and Shannon Holland for putting together the automated tools and easy to use interface allowing non-photogrammetrists to create accurate photogrammetry derived products. REFERENCES De Venecia, K., Miller, S., Pacey, R., and Walker, A. S. (1996). Experiences with a Commercial Package for Automated Aerial Triangulation. Proceedings of ASPRS/ACSM Annual Conference, Volume 1, pp Forstner, W. and Gulch, E. (1987). A fast operator for detection and precise location of distinct points, corners and centers of circular features, ISPRS Intercom. Workshop, Interlaken, pp Lofy, B. (2000). High Accuracy Registration and Targeting. Proceedings of the IEEE - 29 th Applied Imagery Pattern Recognition Workshop, (October 2000), pp Olander, N. F. and Walker, A. S. (1998). Modeling spaceborne and airborne sensors in software. In: International Archives of Photogrammetry and Remote Sensing. Cambridge, England. Vol. 32 Part 2, pp Zhang. B. and Miller, S. (1997). Adaptive Automatic Terrain Extraction. Proceedings of SPIE, Volume 3072, Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision (edited by D. M. McKeown, J. C. McGlone and O. Jamet). pp Zhang, K., Chen, S., Whitman, D., Shyu, M., Yan, J., and Zhang, C. (August 2002). A Progressive Morphological Filter for Removing Non-Ground Measurements From Airborne LIDAR. Journal of LATEX class files, Vol. 1, No. 8.