A FULLY INTEGRATED SYSTEM FOR RAPID RESPONSE INTRODUCTION

Transcription

1 A FULLY INTEGRATED SYSTEM FOR RAPID RESPONSE Alan W.L Ip, Geomatic Analyst Dr. Mohamed M.R. Mostafa, Chief Technical Authority Applanix, A Trimble Company 85 Leek Cr., Richmond Hill Ontario, Canada, L4B 3B3 aip@applanix.com mmostafa@applanix.com ABSTRACT The DSS 322 (Digital Sensor System) is the next generation medium-format, digital aerial camera system with a direct georeferencing capability designed for streamlined airborne digital image acquisition. As a fully integrated system, the all digital workflow provides Plan-Fly-Produce functionality. Combining with its direct georeferencing capability, the requirement for ground control is eliminated. This is a significant cost saving approach to the traditional aerial survey using film cameras, reducing overall project time and expense on field survey and data processing operations. In addition to generating single image digital products, DSS 322 has a full stereo image capability for standard photogrammetric applications requiring a 60% forward lap and 30% side lap. The DSS 322 provides end-to-end digital solutions from rapid response / disaster management to single photo orientation, corridor surveys and photogrammetric mapping. This paper demonstrates the DSS 322 s productivity and efficiency in producing delayed real-time and post-processed orthomosaic imagery. INTRODUCTION Released in 2002, the Applanix Digital Sensor System (DSS) is the first commercial medium-format, digital aerial camera system fully integrated with state of art Direct Georeferencing (DG) and flight management system technology As a mix of modified off-the-shelf and custom components, it provides a fully operational, fully integrated all digital multi-sensor system for digital mapping data acquisition and processing. The DSS 322 is the latest generation DSS model, Similar to traditional aerial survey system using film camera, the DSS is geometrically and radiometrically calibrated (Mostafa and Hutton 2005). In addition, the POSTrack direct georeferencing Flight Management System (FMS) is fully integrated into the DSS system and therefore specific geospatial products can be produced at a low cost and high efficiency, even in a pilot-only flight operational mode. As part of a complete mapping solution to produce orthophoto imagery, the DSS uses two different workflow options. For rapid response applications, RapidOrtho TM images can be produced using the logged real-time POS AV solution and an existing or constant DEM, either while still in the air, or immediately upon landing. This super fast turn around solution further improves the existing orthophoto production workflow (Ip et al, 2004). For full photogrammetric applications with 60% forward and 30% side overlap stereo collection, Applanix integrates the INPHO suite of software tools as part of the production workflow to produce High Precision Post Processed Orthophotos. Here, in addition to the existing Applanix CalQC TM quality control assurance module, the INPHO Match-T, DTMaster and OrthoMaster are used to provide Digital Terrain Model (DTM) self-extraction and quality control functions followed by orthorectification,. As the final step, OrthoVista can be used to generate full radiometrically corrected mosaics from the outputs of RapidOrtho TM or OrthoMaster, generating solutions for different types of applications. In the following sections, the DSS 322 design is first reviewed, followed by geometric analysis of the system. At the end a demonstration and efficiency analysis of the Delayed Real-Time Orthomosaic production using the RapidOrtho TM and INPHO modules is presented.

2 Figure 1. The DSS 322 THE DSS 322 SYSTEM DESIGN From the first release of the DSS in 2003, different components of the DSS have evolved in order to continue satisfying the demands of the airborne environment. Part of this evolution includes the new Applanix AeroLens TM. Manufactured by Carl Zeiss, and featuring the Distagon T optic technology, the lens are exclusively manufactured for the DSS 322 and are the first commercial, medium-format lens specifically designed for airborne applications. As part of the lens system, the lens mount is custom made to maintain the calibrated focus with no moving parts, while the easy switch capability continue to allow lens change between the standard 60 mm AeroLens featuring 44.2 deg cross-track FOV (field of view) and the optional 40 mm lens (61.9 deg cross-track FOV). Figure 1. The Applanix AeroLens by Carl Zeiss In additional to the camera component, the new direct georeferencing component, the POSTrack Flight Management System (FMS) is also included in the DSS 322 system. Integrating the FMS functionality into the DSS, the system can be controlled by a single pilot display, fulfilling the Plan-Fly-Produce functionality in pilot only operation. Furthermore, the POSTrack replacement decreases the DSS system weight by 19%. This weight loss provides more opportunity of the DSS system to be easily deployed in small aircraft and helicopter platform. As a light weight and ruggedized system, the DSS 322 can be integrated with different sensor and installed to fulfill special applications. As a standard DSS installation, it is configured with landscape mode across-track. This minimizes number of strip in a photogrammetric image block setup with 60% forward and 30% side overlap for stereo mapping and DTM self-extraction. However, for corridor mapping application when side overlap is not required, the DSS can be installed such that the landscape mode is along the flight direction, enabling lower flying heights at faster speeds. When the DSS is integrated with LIDAR, the collected DTM further minimizes the forward overlap to about 20% (radiometric correction and safety zone). This type of special configuration may not be available in other large format digital mapping system but it is considered as part of the DSS system design. Table 1 summarizes the specification of the DSS 322

3 Table 1. The DSS 322 Specifications Array size Pixel Size Sensor Head Applanix AeroLens TM by Carl Zeiss Shutter Speed Max Exposure Rate Ground Sample Distance Positioning accuracy (Post-processed) FMS and Direct Georeferencing Operating System Storage Weight 4092 (along flight line) x 5436 (across flight line) 9 micron Proprietary CCD mount and Ruggedized exoskeleton Standard: 60mm, 44 deg FOV Color & CIR Optional: 40mm, 62 deg FOV Color & CIR 1/125 1/4000 sec 2.5 +/ sec 1 sigma 0.03 to 1 meter (platform dependent) 0.05 m horizontal, 0.30 m vertical Applanix POSTrack Embedded OS on flash media 160 GB removable HD & pressurized data brick (~ 3500 images each) Camera w/o Azimuth Mount: 7 kg Azimuth Mount: 14 kg Computer: 34 kg THE DSS 322 SYSTEM CALIBRATION The DSS calibration has been discussed in several papers and it continues be handled by Applanix since no agency has been established for the digital camera calibration. The in-house calibration procedure starts with the terrestrial method using surveyed cage image being taken from several angles. This precisely computes the camera interior orientation parameters (focal length, lens distortions and principal point positions) and the IMU/camera boresight angles are precisely simultaneously (Fraser, 1997 and Mostafa and Schwarz, 2001). On the other hand, the radiometric calibration of the DSS is based on Applanix s TruSpectrum TM technology to produce the best possible image quality from the DSS. A series of correction parameters are applied through the TruSpectrum process, including white balance, lens fall-off, shutter speed, dark noise, CCD gain, CCD flaw correction, Bayer array interpolation and color space correction. As part of the image enhancement, gamma correction, histogram optimization, lens distortion correction, and image sharpening can also be applied. All these correction are performed through the Image View module in the DSS Tools, which is included as a standard application for every DSS system. As the final step of the system calibration, an airborne calibration is performed over a calibration range once the DSS system is installed on the aircraft. This procedure refines the calibration parameters and verifies the specified system accuracy through the CalQC TM module (Mostafa, 2004 and Mostafa and Hutton, 2005). In later section of this paper the geometric system calibration is discussed in some detail through the test data THE DSS 322 DIGITAL ORTHOPHOTO SOLUTIONS The DSS combination of Direct Georeferencing with a Medium-format single frame digital camera has helped to redefine what is now possible with digital orthophotography solutions. Currently the DSS 322 provides end-to-end digital orthography solutions for applications ranging from rapid response / disaster management to corridor surveys and photogrammetric mapping. Such applications have different requirements based on the turn around time and accuracy requirement. Below we define three different types of orthophoto products produced by the DSS that are used in these applications. RapidOrtho TM Fast Orthophoto production is the always ultimate goal for digital solutions, as project cost has traditionally been dominated by data acquisition and processing time, taking weeks or even months to complete a mapping projects. However, in the digital age of airborne mapping using digital frame camera (Mostafa et al, 1997; El-Sheimy, 1996) and the use of direct georeferencing (Schwarz et al, 1993), a complete ortho product can be generated by the DSS in less than 24 hours after landing (Ip et al, 2004a). This provides the opportunity for the use of the DSS system for rapid

4 response application, in which its efficiency has been proved from the hurricane season in 2003 (Isabel) and 2005 (Dennis and Katrina) for damage assessment and ground support. In an effort to improve turn-around time on producing orthophoto products for rapid response applications, Applanix has defined a new DSS 322 product called RapidOrtho TM images. These are orthophoto images created using the logged Real-time POS AV GPS-Aided INS solution and existing DTM or flat earth model. The RapidOrtho TM images are produced during the image development process. The accuracy of the final images is a function of the existing DTM used and the accuracy of the POS AV real-time navigation solution. In the case where a GPS Satellite Based Augmentation Solution (SBAS) is used with the POS AV, a position accuracy of 1 meter RMS or better can be achieved. Optimized with CPU usage in dual processor systems, both image development and orthorectification is asynchronously processed and the performance test of the RapidOrtho TM shows the average time of a full GSD orthophoto can be generated from the raw DSS Color Filter Array (CFA) image in less than 20 seconds. This is a super fast turn-around time for typical mission of 1000 images to be reduced to 5.5 hours or less, instead of 24 hours. Post-Processed Ortho Here we define a Post-Processed Ortho as an orthophoto image produced by the DSS using the post-processed POS AV GPS-Aided INS solution to achieve centimeter position accuracy. This is obtained using the Carrier-Phase Differential GPS (CPDGPS) technique with a devoted basestation or Continuous Operating Reference Station (CORS) data. Processing with the precise system calibration parameters, an orthophoto can be generated with an existing DTM at a minimum overlapping area of about 20% (for radiometric correction and safety zone) in both forward and sideway direction. This is a huge saving in flying time from the photogrammetric image block with 60% forward and 30% sidelap. In compare to the RapidOrtho TM solution, the orthophoto product is further delayed by the post-processing of the navigation solution, which is mainly depends on the basestation data availability. If a devoted basestation is used and it is located in a remote area, additional time will be added as an orthophoto cannot be generated without the basestation being retrieved and processed. Although CORS data can be used as an alternative to a devoted basestation, mapping in remote area can potentially introduce accuracy problem due to long baseline separation. However, under careful project design (minimizing baseline or the use of multi-baseline CDGPS processing) and the DSS system s precise calibration parameters (with proper calibration procedure), the Post-Processed Ortho position accuracy is usually within 2 times of the collected GSD, assuming availability of a suitably accurate DTM. Such accuracy is applicable to GIS and Remote Sensing application where less accurate product is accepted. High Precision Post-Processed Ortho This workflow is dedicated to application that requires the highest accuracy solution. It involves following careful quality control processes designed around the Direct Georeferencing concept embedded in the DSS The first step is to perform proper mission planning to ensure adequate GPS satellite coverage during the mission, and adequate base station separation. Post mission the CalQC TM tool is used to asses the overall quality of the mission using a quality control block. A quality control block can be a subset of the main mission image block or can be flown separately at the beginning or the end of the mission. The QC block should contain a minimum number of 8 photos per strip, collected in opposite direction for at least 3 strips with a minimum overlap of 60% forward and 30% sidelap. Through the calibration process in CalQC TM, mission specific calibration parameters are refined, including IMU/Camera boresight angles and camera interior orientation parameters (focal length and principal point offsets only), minimizing residual error in the system. Figure 2 illustrates the DSS workflow to generate High Precision Post-Processed Orthos. If an existing DTM in high density (e.g. LIDAR data) is not available, self-extraction can be applied on the full stereo image block, collecting detail information of the Terrain. Applanix recommends INPHO s Match-T and DTMaster, providing DTM self-extraction, editing and stereo viewing capability. Following the quality control processes suggested by Applanix, the High Precision Ortho product can achieve position accuracy better than 1 GSD. An alternative method to produce High Precision Post-Processed Ortho is to use Integrated Sensor Orientation. Based on the quality control concept, a mass distribution of tie points on the overlapping area of the entire mission photogrammetric image block is used to refine the exterior orientation parameters with an EO-centric triangulation solution. However this can only work if the entire mission is flown in a block configuration with 60% endlap and 30% sidelap, which in many applications is not efficient.

5 Figure 2. Illustration of the High Precision Post-Processed DSS workflow PERFORMANCE ANALYSIS OF THE DSS 322 In this section the geometric performance of the DSS 322 is first discussed through the pre-mission calibration flight collected over Shelby County airport, Alabama This is then followed by a demonstration of a RapidOrtho TM production project. Geometric Performance Analysis The pre-mission calibration flight of the DSS 322 was flown by AerialExperts in early July, This flight had a total of 6 strips collected at 2 different altitudes. The flight configuration parameters are listed in Table 2, and Figure 3 shows the flight layout of the image block and the orthomosaic of the area generated using the INPHO OrthoBox modules with existing DTM data (NED at 1/3 resolution) downloaded from USGS. Through out the calibration process, all tie/pass points are generated automatically using the ATG module in CalQC TM, and the only manual process is the measurement of the check points. Note that, in Figure 3, a triangle represents Check Point, a cross represents a photo centre, and a dot represents a tie/pass point. Table 2. Shelby, Alabama flight configuration Parameter Value Remarks Flight Altitudes (m) 1050 and 1750 for calibration purpose GSD (m) 0.16 and 0.26 Focal Length 60 mm non-calibrated No. of Photos 41 No. of Strips 6 in 4 directions

6 Figure 3. Calibration flight configuration (left), Orthomosaic of the flight area (right) The calibration parameters of the DSS 322 system were derived using the above image block and the results are listed in Table 3. Examining the calibration parameters, the boresight RMS is clearly within the POS AV system specification, which is a factor of the system orientation performance. Similar performance is also shown in the Exterior Orientation (EO) RMS. Through the RMS of the image coordinate residuals, the tie point collected in both STG and ATG module is at the order of ½ pixel or better. Further examining the calibration result, there is a potential shift in the Y component, at -0.48m. Carrying the 7-parameter transformation (datum shift calibration) in CalQC TM, the mean value on the Y component becomes 0.19m. Although the datum shift determination is less important in this pre-mission calibration, this presents the capability of the CalQC TM module for handling mission specific parameters Table 3. Calibration report for the Shelby, Alabama flight EO RMS X/Tx Y/Ty Z/Tz Value RMS XYZ (m) (deg) Image RMS ( m) 4 4 Checkpoint Residuals (m) Mean STD DEV RMS Checkpoint Mean Residuals STDEV (w/datum) RMS

7 RapidOrtho TM Productivity Analysis To demonstrate the speed at which RapidOrtho TM images can be produced by the DSS, a recently collected Tampa Bay flight from Florida is used. The time to generate the RapidOrtho TM images using the logged real-time POS AV solution was recorded, and this was compare with the time to produce the High Precision Post-Processed Orthophoto images using the method described in Figure 2. At the time of data processing, the project area has not yet been surveyed and therefore no ground control point is available to perform geometric analysis of the data. The project info about the Tampa Bay flight is listed in Table 4 and the orthomosaic of the full project is shown in Figure 4 (red dot represents a photo center). Table 4. Flight configuration of the Tampa Bay flight No. Photos 355 No. Strips 24 Area of coverage ~ 270 km 2 Flying Altitude 2150 m GSD 0.32 m Forward Overlap 20 % Side Overlap 20 % Flying time 3.5 hrs Figure 4. Flight trajectory of the Tampa Bay flight overlaying onto the orthomosaic The northern part of the flight was selected to perform an expanded quality control procedure using CalQC TM for the High Precision Post-Processed Orthophoto process. It consists of a total of 117 photos, with 13 photos per strip. Although there is no existing ground control to completely analysis the mission-specific calibration parameters such as the local datum shift, the result still provides important information about the stability of the system such the IMU/Camera boresight angles. The QC block is further divided in sub-blocks to understand the repeatability of such parameters during the flight, and the sub-block locations are defined as in Figure 5. Each sub-block consists of 3 strips with 8 photos per strip, and it can be seen from the orthomosaic that they consist of different kind of terrain features. The resulting boresight value from each sub-block is presented in Table 5.

8 Figure 5. QC subsection block configuration (left), orthomosaic of the subsection (right) Table 5. Boresight values for each sub-block after quality control procedure TL (Top Left Corner) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 4 5 TC (Top Center) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 2 2 TR (Top Right Corner) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 2 2 LL (Lower Left Corner) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 4 6 LC (Lower Center) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 3 3 TL (Top Left Corner) X/Tx Y/Ty Z/Tz Value RMS Image RMS ( m) 2 2 The above results show repeatability of similar values over the whole sub-section of the Tampa Bay flight, having the RMS value within the specification of the DSS system.

9 Table 6 lists the amount of processing time towards each stage of the orthophoto production using the collected GSD value. The workstation is a Pentium 4 single Processor running at 2.2 GHz with 2GB of memory (faster times are possible on a faster workstation). Table 6. Processing time of the Tampa Bay Flight Time Elapsed Delayed Rapid Ortho High Precision / Post-Processed Ortho Data Collection (hrs) 3.6 hrs X Delayed Real-Time Navigation Solution 2 mins X Post Processed Navigation Solution (using CORS data) Quality Control (using 3x8 QC block and including image preview generation) Ortho production using RapidOrtho Asynchronous Synchronous Tiled and Radiometric balanced Orthomosaic production using OrthoVista 60 mins X 45 mins X 60 sec / photo 90 sec / photo 6 hours X Total 9.6 hrs 20.2 hrs X X From Table 6, the efficiency of producing the RapidOrtho TM images can be easily understood. During the workflow, RapidOrtho TM images can be processed in asynchronous mode, delivering image processing and orthorectification in a parallel. However, in High Precision Post-Processed Ortho workflow, developed images are first used for quality control procedure and therefore orthorectification is delayed as a sequential step, resulting in a 50% more processing time on the image itself. On the other hand, the RapidOrtho TM image production is usually focused on smaller format of the images, which makes them easy to be handled and disseminate across the web. This usually bypasses the final radiometrically balancing of the orthomosaic which takes approximate 1 minute per image at a 20% overlap. Such a step is however essential for the Post-Processed Ortho workflow in which the final product is used for GIS and remote sensing applications. CONCLUSION The DSS 322 is a versatile solution that can produce a variety of orthophoto products for many different applications. The RapidOrtho TM images provide a super fast turn around solution for rapid response and emergency application. In this case, orthophotos can be generated at a rate of 20 sec per photo under a dual processor workstation in asynchronous processing mode, at an accuracy of 1 m level or better when using SBAS. For highest accuracy but longer lead-time products, the High Precision Post-Processed Ortho process is the complete solution. Although the Tampa Bay flight shows the radiometric balanced orthomosaic can be generated within 24 hours, this does not account for the in-depth quality control procedures such as DTM self-extraction and /or editing from LIDAR data. These additional steps are processing power dependent, but careful project planning is also required, including overlapping percentage and base station selection. This can easily increase the data acquisition time and delay the final product to be generated. REFERENCES Braun, J., (2003). Aspects on True-Orthophoto Production. Proceedings of the 49th Photogrammetric Week, Stuttgart, Germany, September Grejner-Brzezinska, D.A., (2001). Direct Sensor Orientation in Airborne and Land-based Mapping Application. Report No. 461, Geodetic GeoInformation Science, Department of Civil and Environmental Engineering and Geodetic Science, The Ohio State University. El-Sheimy, N., (1996). The Development of VISAT-A Mobile Survey System for GIS Applications, UCGE Report #20101, Department of Geomatics Engineering, The University of Calgary, Canada.

10 Fraser, C.S., (1997). Digital Camera Self Calibration, ISPRS Journal of Photogrammetry & Remote Sensing, 52(1997): Heipke, C, Jacobsen, K, Wegmann, H, Andersen, O and Nilsen, B., (2000). Integrated Sensor Orientation-An OEEPE Test. IAPRS, Vol. XXXIII, Amsterdam, Ip, A.W.L., M.M.R. Mostafa, and N. El-Sheimy, (2004a). Fast Orthophoto Production Using The Digital Sensor System. Proceedings of the 7th Annual International Conference - Map India 2004, New Delhi, India, January Ip, A.W.L., N., El-Sheimy and J., Hutton (2004b). Performance Analysis of Integrated Sensor Orientation, International Archives of Photogrammetry and Remote Sensing, ISPRS Comm. V, Vol. XXXV, Part B5, pp , Istanbul, Turkey, July 12-23, Jacobsen, K, (2002). Calibration Aspects in Direct Georeferencing of Frame Imagery. Presented at the ASPRS Fall Meeting (PECROA) Denver, Colorado, USA. Mills, J.P., I. Newton, and R.W. Graham, (1996). Aerial Photography for Survey Purposes with a High Resolution, Small Format, Digital Camera Photogrammetric Record 15 (88): , October, Mostafa, M.M.R., and J. Hutton, (2005). A Fully Integrated Solution for Aerial Surveys: Design, Development, and Performance Analysis, PE&RS, 71 (4): Mostafa, M.M.R., (2003). Design and Performance of the DSS. Proceedings of the 49th Photogrammetric Week, Stuttgart, Germany, September 1-5, Mostafa, M.M.R., (2002). Camera/IMU Boresight Calibration: New Advances and Performance Analysis, Proceedings, ASPRS Annual Meeting, Washington, D.C., April Mostafa, M.M.R. and K.P. Schwarz, (2001). Digital image georeferencing from a multiple camera system by GPS/INS. ISPRS Journal of Photogrammetry & Remote Sensing 56 (2001): Mostafa, M.M.R., J. Hutton, (2001). Direct Positioning and Orientation Systems: How Do They Work? What is The Attainable Accuracy? Proceedings, The American Society of Photogrammetry and Remote Sensing Annual Meeting, St. Louis, MO, USA, April Mostafa, M.M.R. and K.P. Schwarz, (2000). A Multi-Sensor System for Airborne Image Capture and Georeferencing. PE&RS, 66 (12): Mostafa, M.M.R., K.P. Schwarz, and P. Gong, (1997). A Fully Digital System for Airborne Mapping, KIS97 Proceedings, Banff, Canada, June 3-6, pp Schwarz, K.P., M.A. Chapman, M.E. Cannon and P. Gong, (1993). An Integrated INS/GPS Approach to The Georeferencing of Remotely Sensed Data, PE&RS, 59(11): Schwarz, K. P., (1995). Integrated Airborne Navigation Systems for Photogrammetry. Proceedings of Photogrammetric Week 95, Eds. D. Fritsch/D. Hobbie, Wichmann, Heidelberg, pp Škaloud, J., (1999). Problems in Direct-Georeferencing by INS/DGPS in the Airborne Environment. ISPRS Commission III, WG III/1 Barcelona, Spain, November Toth, C. and D.A. Grejner-Brzezinska, (1998). Performance Analysis of The Airborne Integrated Mapping System (AIMSTM), International Archives of Photogrammetry and Remote Sensing, 32 (2): White, S. and M Aslaksen, (2006). NOAA s Use of Direct Georeferencing to Support Emergency Response. PERS Direct Georeferencing Column, June, 2006Mostafa, M.M.R., Design and Performance of the DSS. Proceedings of the 49th Photogrammetric Week, Stuttgart, Germany, September 1-5, 2003 Yastikli, N., (2004), The Effect of System Calibration on Direct Sensor Orientation, International Archives of Photogrammetry and Remote Sensing, ISPRS Comm. I, Vol. XXXV, Part B1, pp , Istanbul, Turkey, July 12-23, 2004.