Mobile Mapping - An Emerging Technology For Spatial Data Acquisition

Transcription

1 Mobile Mapping - An Emerging Technology For Spatial Data Acquisition Dr. Rongxing Li Associate Professor Department of Civil and Environmental Engineering and Geodetic Science The Ohio State University Columbus, OH ABSTRACT Mobile mapping has been the subject of significant research and development by several research teams over the past decade. A mobile mapping system consists mainly of a moving platform, navigation sensors, and mapping sensors. The mobile platform may be a land vehicle, a vessel, or an aircraft. Generally, the navigation sensors, such as GPS (Global Positioning System) receivers, vehicle wheel sensors, and INS (Inertial Navigation System), provide both the track of the vehicle and positional and orientational information of the mapping sensors. Objects to be surveyed are sensed directly by mapping sensors, for instance CCD (Charge Coupled Device) cameras, laser rangers, and radar sensors. Since the orientation parameters of the mapping sensors are estimated directly by the navigation sensors, complicated computations such as photogrammetric triangulation are greatly simplified or avoided. Spatial information of the objects is extracted directly from the georeferenced mapping sensor data by integrating navigation sensor data. Mobile mapping technology has evolved to a stage which allows mapping and GIS industries to apply it in order to obtain high flexibility in data acquisition, more information with less time and effort, and high productivity. In addition, a successful extension of this technology to helicopter - borne and airborne systems will provide a powerful tool for large scale and medium scale spatial data acquisition and database updating. This paper provides a systematic introduction to the use of mobile mapping technology for spatial data acquisition. Issues related to the basic principle, data processing, automation, achievable accuracies and a break down of errors are given. Application considerations and

2 application examples of the technology in highway and utility mapping are described. Finally, the perspective of the mobile mapping technology is discussed. INTRODUCTION Large scale spatial data and associated attributes in GIS (Geographical Information Systems) are in high demand in order to generate new databases and to update existing databases in applications such as transportation, utility management, and city planning. The acquisition of such information is mostly realized by aerial photogrammetry or terrestrial surveying using, for example, total stations. In the former case, aerial photographs may not provide sufficient detailed information of planimetric and horizontal object features, for example manholes, positions of road curbs and center lines, building facades etc., because of the photoscale and perspective projection geometry. In the latter case, object features can be measured very accurately. However, the operational time of the field survey and associated costs are major concerns and sometimes make a project impossible. Furthermore, a road or utility survey along a highway may be costly or impractical if the traffic has to be stopped or detoured for a long period. However, a mobile mapping system, equipped with a mobile platform, navigation sensors, and mapping sensors, is capable of solving the above problems. The mobile platform may be a land vehicle, a vessel, or an aircraft. Generally, the navigation sensors, such as GPS (Global Positioning System) receivers, vehicle wheel sensors, and INS (Inertial Navigation System), provide both the track of the vehicle and positional and orientational information of the mapping sensors. Objects to be surveyed are sensed directly by mapping sensors, for instance CCD (Charge Coupled Device) cameras, laser rangers, and radar sensors. Since the orientation parameters of the mapping sensors are directly supplied by the navigation sensors, complicated computations such as photogrammetric triangulation are reduced or avoided. Spatial information of the objects is extracted directly from the georeferenced mapping sensor data by integrating navigation sensor data. Advantages of such a system include: Increased coverage capability, rapid turnaround time, and thus, improved efficiency of the field data acquisition; Integration of various sensors so that quality spatial and attribute data can be acquired and associated efficiently; Simplified geometry for object measurements supported by direct control data from navigation sensors; Flexible data processing scheme with original data stored as archive data and specific objects measured at any time; and

3 Strongly georeferenced image sequences which provide an opportunity for automatic object recognition and efficient thematic GIS database generation. Mobile mapping technology has been researched and developed since late 1980's. It was inspired by the availability of GPS technology for civilian uses. Experiments have been made to strengthen the geometry and to reduce the ground control in photogrammetric aerotriangulation by GPS observations (van der Vegt 1988, Colomina 1989, Jacobsen 1993, Ackermann and Schade 1993, Merchant 1994). The mobility of the mapping systems has always been a concern. It requires high resolution mapping sensors covering large areas, and high accuracy navigation sensors determining the positions and orientations of the vehicle. The integration of the various sensors involved, such as GPS, INS, laser scanners, CCD cameras and others in a mapping system has been investigated and demonstrated by Hein (1989), Krabill (1989), Schwarz et al (1993b), Scherzinger et al (1995), Da and Dedes (1995), and Schwarz and El-Sheimy (1996). In some applications, only GPS is used to provide the dynamic positions of the vehicle and the locations of the mapped objects. For example, an all-terrain vehicle equipped with a GPS receiver gives the shoreline position when it moves along the water mark line (Shaw and Allen 1995). Video images or other GIS attributes can be provided with associated locations surveyed by GPS receivers in a static or kinematic mode. There are a number of alternatives for determining the attitude of a vehicle including dead reckoning systems (Novak 1995, Da and Dedes 1995), gyroscopes (Cosandier et al 1993), INS (Wei and Schwarz 1990), and the attitude information derived from multiple GPS antennas (Lu 1995, Sun 1996). These techniques offer different accuracies with varying costs. More sophisticated applications of integrated sensors have been researched recently. For instance, positions determined by a GPS receiver on a car are integrated with a road network database to support car navigation (Krakiwsky and French 1995). A GPS assisted Autonomous Precision Approach and Landing System (APALS) uses calibrated Synthetic Aperture Radar (SAR) maps to support aircraft landing (Cavanaugh 1995). Positional and orientational parameters are derived from GPS and INS data to control sonar beams on a vessel and aerial cameras (Schwarz et al 1993b, Scherzinger et al 1995). Land vehicle based mobile mapping systems result in, among others, a) close distances between the systems and the objects to be surveyed, b) no ground control and no triangulations across images exposed at different time, and c) completely digital processing. These systems are designed mainly for mapping purposes. Different approaches to the system design and implementation have been used. To give a few examples, TruckMapTM (Pottle 1995) uses DGPS, a reflectorless laser rangefinder and a high accuracy azimuth engine to establish object positions. Video images are acquired, each image being associated with a digitized position and time reference (Pottle 1995, Chapman and Baker 1996). GPS-VanTM (Bossler et al 1991) started with DGPS, CCD cameras, an odometer for measuring the distance driven, a wheel sensor for deriving vehicle speed, and a gyro system for obtaining orientation parameters. The system was improved and successfully employed, for instance, in railway-related mapping (Novak 1995, Bossler and Toth 1996). VISAT-Van (Schwarz et al 1993, Li et al 1994) uses a high accuracy strapdown inertial navigation system to provide high quality angular orientation parameters of the CCD cameras, along with positional orientation parameters from DGPS. The

4 new configuration of the system contains eight CCD cameras covering a view field of 180o. Similar configurations were implemented and reported in KiSS (Hock et al 1995) and GPSVision (He 1996). Processing of the vast amount of mobile mapping data is subsequently a very important task. So far, there is no common commercial software capable of handling the data from different mobile mapping systems. Automation of the procedures of mobile mapping data processing has not been extensively researched since most efforts seem to have been made in the development of the data acquisition systems. However, the automation of processing such large observation databases is of great importance. Automatic matching of corresponding image points in an image sequence was reported by Li et al (1994) and Xin (1995). Extraction of road center lines and curb lines from mobile mapping image sequences was researched by He and Novak (1992), Tao et al (1996) and Li et al (1996a). Three-dimensional coordinates in the object space calculated from the mobile mapping image sequences can be optimized by considering both the precision and reliability (Li et al 1996b). A discussion of building object - oriented 3D databases from mobile mapping data can be found in Qian (1995). Several land vehicle based systems have been developed. Efforts in realization of the concept in the airborne environment have been made by researchers (Lapine 1991, Merchant 1994, Bossler 1996). This paper introduces the mobile mapping technology for spatial data acquisition systematically, including its its principle, data processing, and automation. Achievable accuracies of the current mobile mapping systems and a break down of errors are given. Examples and considerations relating to the use of the technology in highway and utility applications are described. The perspective of mobile mapping technology is also discussed. Mobile Mapping Technology Spatial and Time Referencing in Mobile Mapping Systems The position of the moving platform changes dynamically along a predefined track to acquire mapping data of objects within its field of view. Two critical issues are: a) how to determine the dynamic positions of the platform itself at any time, and b) how to further derive the spatial information of the objects of interest in the field of view from the platform. Without appropriately defined spatial and time reference systems, it is difficult to describe relationships between the objects, the sensors, the platform, and the world. The track of the vehicle is usually defined in a global coordinate system (Figure 1). Depending on the geographical extent of a project and the application requirements, this global coordinate system may be, for example, the Universal Transverse Mercator (UTM) projection system, a state plan coordinate system, or a 3- D Cartesian coordinate system. The platform coordinate system (XV-YV-ZV) is defined on the vehicle and used to integrate various sensors such as the GPS receiver, INS unit, laser device, and cameras. An individual sensor, say i-th sensor, has its own local coordinate system (xi-yi-zi),

5 which is related to the platform coordinate system. The platform coordinate system is further referenced to the global coordinate system. For example, the object to be surveyed is a traffic sign which is within the field of view of the i-th sensor (a camera in Figure 1). The objective is to derive the coordinates of the traffic sign in the global coordinate system (Xj, Yj, Zj). This can be realized by detecting the object using a single or multiple sensors and calculating its coordinates in a local coordinate system. By means of the platform coordinate system, the coordinates of the object in the local sensor coordinate system are transformed to the global coordinate system. The mathematical models for calculating the object coordinates in the local coordinate system depend on the types of sensors. This will be discussed in a later section. The determination of transformation parameters between the local sensor coordinate systems and the platform coordinate system is usually carried out in a system calibration procedure. This procedure requires a special setting of precisely surveyed and well distributed control points, and has to be performed periodically, or whenever there is a change in the relationship between the sensors (Moffitt and Mikhail 1980, El-Sheimy and Schwarz 1993). A time reference system is used to track the kinematic positions of the platform. It is also used in the synchronization of the sensors to incorporate signals from various sources taken at different epochs. The signals are both spatially and temporally referenced to derive the global spatial information of the object detected by the sensors. The time reference system used in mobile mapping is dependent on the sensors used and the accuracy required. GPS time signals are related to UTC (Universal Coordinated Time) which are available in received GPS data (Leick 1995). INS, however, provides more frequent time updates (El-Sheimy and Schwarz 1993). Once a time system is chosen as the time reference system, signals from different sources can be integrated and the kinematic platform locations can be determined. Kinematic Mapping Principle GPS technology is commonly employed in the precise determination of the dynamic positions of the platform. The positions of the objects to be measured are then acquired using the mapping sensors. Mobile mapping systems are categorized into direct measurement systems and indirect measurement systems according to the methods in which the objects are measured. A direct measurement system collects object positions directly using the GPS/DGPS measurements. The simplest system of this kind uses a GPS receiver moved from one object point to the other. The coordinates of the point J (rgj in the global coordinate system) are measured and the attributes are associated in real-time. To achieve a reasonable accuracy for mapping (Figure 2), DGPS is applied by adding an additional GPS receiver at a master station with the known coordinates in the global coordinate system (rgms). The object position rgj is

6 computed by rgj = rgms + rgpsj, (1) where rgpsj is provided by the DGPS result. A sophisticated direct measurement system determines point positions based on "continuous" observations along the vehicle track. More accurate geometric information can be achieved by post processing algorithms. For example, the OTF (On-The-Fly) algorithm for kinematic positioning allows the vehicle to travel at a speed of 60km/h and to reach an accuracy of centimeters provided that the baseline between the rover and the master receiver is less than about 50km (Schwarz et al 1993a, Cannon 1994, Leick 1995). Similar results in airborne applications at a speed in excess of 360km/h were demonstrated (Lapine 1991). The direct measurement system can be applied to many situations. However, if a digital terrain model is to be generated, there must be a very dense network consisting of a large number of vehicle tracks to cover the area. This may not be a practical and efficient method for this purpose. In an indirect measurement system, GPS is employed to control other mapping sensors which acquire data of objects to be surveyed. A typical example of this kind of system is a van equipped with a GPS receiver, a pair of CCD cameras, and orientational sensors (Figure 3). These sensors on board are mounted on a "rigid" body. They are associated with the platform coordinate system (XV-YV-ZV). In some systems, the platform coordinate system is, for example, defined as INS frame (El-Sheimy and Schwarz 1993). The orientation of the platform in the global coordinate system is represented by a rotation matrix MVG which is a function of heading, pitch, and roll. These three angles can be measured dynamically by the orientation sensors, for example, by means of INS, gyroscopes, or multiple GPS antennas. The system calibration supplies the offset of the GPS antenna rgpsv and those of the camera exposure centers rvc1 and rvc2 in the platform coordinate system. The objective is to compute the position of point J (rgj) in the global coordinate system. At any time, DGPS provides the position of the receiver along the track, rggps in the global coordinate system. Thus, the origin of the platform coordinate system is rgv = rggps + MVG rgpsv. (2) The positions of the two exposure stations are rgc1 = rgv + MVG rvc1 rgc2 = rgv + MVG rvc2. (3)

7 A stereo image pair from the two cameras forms a stereo model in the local sensor coordinate system defined by the relative positions and orientations of the two cameras. Any object J in the overlapping area of the stereo images can be measured in the image space, and its position in the local sensor coordinate system can be computed as rc1j and rc2j. Finally, the position of point J in the global coordinate system is calculated as rgj = rgc1 + MVG MC1V rc1j or rgj = rgc1 + MVG MC2V rc2j, (4) where MC1V and MC2V are rotation matrices between the two local sensor coordinate systems (Camera I and Camera II) and the platform coordinate system. The solutions of equations (1), (2), and (3) establish direct estimates of the exterior orientation parameters of the cameras based on GPS, INS, and calibration measurements. The solution of equation (4) requires calculation of either rc1j or rc2j. These vectors are the photogrammetric measurements which will be discussed in equation (5). Additional mapping sensors may be integrated into the system. For instance, laser devices can be used to scan road surfaces and to acquire road surface data. Video cameras can be employed to record analog images and provide continuous video images. In Equations (1) to (4), observations of DGPS and orientational sensors provide the dynamic positional and orientational link between the platform and the global coordinate system rggps and MVG. In addition, with the help of the calibration parameters, positions of the objects are measured in the image space and transformed to the global coordinate system. In this way, no ground control points and terrestrial photogrammetric (strip) triangulations are required. Since the view fields of mapping sensors - cameras in this case - cover a much wider area, especially in the cross track direction, the indirect measurement system usually has a higher efficiency of data collection for the same track line in comparison to a direct measurement system. Mobile Mapping vs. Real-Time Mapping The objective of mobile mapping is to acquire data for deriving spatial and attribute information digitally and dynamically during the course of surveying. Data processing and production of GIS databases can be carried out in a post processing procedure. On the other hand, real-time

8 mapping requires that the products, such as maps or GIS databases, be delivered during the course of surveying. In many applications the post processing does not cause any problem. For example, it is important to reduce the field survey time when a utility survey along a highway is conducted. However, in some other applications, the post processing becomes an obstacle, when, for example there is no previously acquired spatial data available and the mobile mapping system is employed to produce the spatial information used immediately. Such applications can be found in military situations and in emergency response systems. In the latter cases, real-time mobile mapping is unavoidable. Currently, there are three major factors affecting real-time mobile mapping: a) Differential GPS corrections are not available until observations at both the master station and rover stations are postprocessed, so that the determination of the vehicle positions by DGPS cannot be performed. Transmission of the differential GPS corrections between the master station and rovers by radiobeacons has been experimented by U.S. Coast Guard (Leick 1995). This makes real-time kinematic positioning possible. If high quality corrections for OTF (On-The-Fly) are transmitted, real-time mobile mapping will have one less obstacle. b) Integration of data sets from different sensors sometimes requires accumulative data acquired over a period of time instead of in a moment. An extreme example is when INS data captured within a tunnel have to be integrated with GPS data at the two ends of the tunnel. c) Although some simple features, such as the track of the vehicle, some marked targets, and road centerlines, can be extracted and their positions in the object space can be calculated in a relatively short time (He and Novak 1992, Li 1993, Li et al 1996), measurements of most features require either an interactive or semiautomatic procedure which cannot usually be performed in real-time. Mobile mapping systems may provide some simplified real-time functions, for instance for checking completeness of the data acquired. Data processing and measurements are conducted in post processing sessions. Extraction of Spatial Information Derivation of Spatial Information Spatial information from a direct measurement system is primarily obtained from the positions of the receiver. In this paper, methods for deriving spatial information from indirect measurement systems are discussed. If CCD cameras are employed (Figure 3), objects to be measured are identified in a pair of stereo images acquired by the cameras. The objective is to measure the objects in the images and to derive their positions in the global coordinate system. Suppose that a point J appears in the left and right images with its image points as j and j', respectively. Their coordinates in the images are measured as (xj, yj) and (xj', yj'). According to photogrammetric principles (Moffitt and Mikhail 1980), three vectors, namely rc1j, rc2j and B, shall be on the same plane. This coplanarity condition can be expressed as a scalar triple product equation

9 ( rc1j x rc2j ) B = 0 (5) This equation can be elaborated to a function of the observations (xj, yj) and (xj', yj'), the calibration parameters, and the unknown coordinates of point J in the platform coordinate system. The calibration parameters include focal lengths of the cameras, the rotation matrices MC1V and MC2V from the local camera coordinate systems to the platform coordinate system, and the coordinates of the camera exposure centers in the platform coordinate system. As a result of Equation (5), rc1j and rc2j are computed and used in Equation (4) to calculate the position of point J in the global coordinate system. It should be noted that the parameters in and the result of Equation (5) are all relative to the platform. Only if the dynamic position and attitude of the platform in the global coordinate system are measured by DGPS and by orientational sensors, can the relative position of point J be transformed to the global coordinate system using Equation (4). Since the images are taken in sequence, one object point is usually covered by multiple images which may form stereo image pairs with combinations of images taken at different epochs. A simple way to improve the accuracy is to measure the same point in all possible images for calculating the coordinates of the point. This simultaneous approach was proven inefficient in comparison to a sequential estimation algorithm based on Givens transformation (Gruen and Kersten 1995). Further research by Li et al (1996a) led to an algorithm for selecting an optimal image pair from an image sequence for photogrammetric point intersection using Kalman filtering. This algorithm optimizes both precision and reliability. Image matching techniques automate the procedure for measuring point and line features. Although not all features can be measured automatically at the moment, the productivity can be improved by the matching techniques, considering the sizes of large databases to be generated. Points, arcs and polygons are three primary spatial components of a GIS database. With points derived from the image sequences as discussed above, the other two components can easily be generated by an extended point measurement procedure (Li et al 1994). Derivation of Attribute Information Attribute information is another important data category in a GIS database. It is associated with spatial features to describe additional non-spatial characteristics of objects. Some systems collect

10 attribute data such as recorded voices indicating the wildlife species found in a location, or video images of road surfaces, together with the spatial information acquired, either as points for the species, or as "continuous" lines of the road surfaces in video images. In an indirect measurement system, interactive interpretation of the image sequences is mostly used to derive the attributes. This requires that the mobile mapping software be designed and implemented in such a way that the attributes can be associated with the spatial features during the measurement procedure (Li et al 1994). An ideal way to generate the attribute information is to extract the information from the image sequences automatically (He and Novak 1992, Li et al 1996b, Tao et al 1996). Currently, there are some very limited cases where this automation may be performed successfully. First, text information in images, for example street names on road signs and texts of traffic signs, can be extracted by pattern recognition methods. Depending on the orientation of the sign and of the cameras, the projection of the text on the image may be distorted or even invisible. Robust text recognition algorithms which consider the field conditions should be developed. Second, some specific types of objects, such as manholes and fire-hydrants, have a symmetric geometry in the horizontal plane and their projections in the images are less distorted. Since the geometry of these objects is known, artificial images can be generated. Then these images can be matched with the object features in the real image sequences to find out the desired objects automatically. Finally, a general classification scheme, such as that used in the classification of satellite remote sensing images, is not appropriate for the mobile mapping images, because of the rapidly changing scales of the objects in the images and because of the extremely detailed and diverse information involved. The number of object classes in such an image is much higher than that of satellite imagery. A semiautomatic procedure is practical for GIS database production. For each spatial feature, its geometry can be measured either interactively or automatically using image matching techniques. The attribute is then determined automatically if possible. Otherwise, an interpretation by the operator is necessary. In the current stage, this is still the most effective and reliable method for attribute derivation from the mobile mapping data. Accuracy The accuracy of the coordinates of a point in the global coordinate system derived from the mobile mapping data is one of the important quality measures. The errors involved are in turn contributed by those from the system components. The following discussion focuses on three error categories in mobile mapping systems (Table 1): control data errors, mapping sensor errors, and the overall system error, based on a typical configuration with a land vehicle, DGPS, INS, and CCD cameras.

11 Control data refer to observations from the navigation sensors used to derive the positional and orientational parameters of other mapping sensors such as cameras. Thus, the accuracy of these observations directly affects the estimated positions and attitudes of the mapping sensors. Furthermore, it influences the point positions derived from the mapping sensor data. According to recent reports by Pottle (1995), El-Sheimy (1996), and Bossler and Toth (1996), if DGPS and post processing techniques are employed, the accuracy of the dynamic platform positions can be determined in the range of 6 to 20 cm. The speed of the vehicle is controlled under 70km/hour. Angular parameters derived from a strapdown INS are accurate to within 1 to 6 minutes if the time-dependent drift is corrected (Schwarz and El-Sheimy 1996, Hock et al 1995). This angular error would result in a linear error of 1.5 ~ 8.7 cm at a distance of 50 m from the system. Inexpensive approaches using gyroscopes provide the angular parameters with a lower accuracy of 0.75 ~ 2.0 degrees (Cosandier et al 1993). Experiments calculating the attitude information from data acquired by multiple GPS receivers on a vehicle gave an accuracy of 0.3 ~ 0.5 degree with an average baseline between the receivers of 1.7 ~ 1.0 m (Lu 1995, Sun 1996). It is obvious that at the present time, the angular parameters calculated from the observations of gyroscopes and multiple GPS receivers are not satisfactory for high accuracy control of the mapping sensor. The synchronization of the sensors gives an error of about 1 ~ 2 msec which translates to a linear error of 1.9 to 3.9 cm at a vehicle speed of 70 km/h (Schwarz and El-Sheimy 1996). Errors contained in the mapping sensor data include system calibration errors, image measurement errors, and lens distortion errors (if not corrected) in the photogrammetric processing (table 1). After a system calibration with a well-established control field and an appropriate procedure (El-Sheimy and Schwarz 1993), the calibration errors can be as small as 2 ~ 5 mm. In the image space, locations of objects with clear image features and a symmetric geometry, such as marked targets used in system calibrations, can be determined at a subpixel level, for example 1/20 pixel, using image metrology techniques (Cosandier and Chapman 1992, Li 1993). Measurements of general objects can be performed with an accuracy of 0.5 pixel if a zoom function is employed. The lens distortion is usually modeled and corrected in the photogrammetric processing. If not corrected, this error may reach 1.5 pixels in the image space. The errors caused by imprecise image measurements were reported to be 4 cm for objects within 50 m from the cameras, while the effect of the lens distortion reaches 12.7 cm if not corrected (Schwarz and El-Sheimy 1996). An estimate of the overall system accuracy depends on the accuracy of the individual components of the system. Taking the positional accuracy of an object point in the object space as the measure of the overall system accuracy, some systems have reached 30 ~ 50 cm (both horizontal and vertical) if the objects are 30 ~ 60 m from the system (Schwarz and El-Sheimy 1996, Bossler and Toth 1996, Hoch et al 1995). This overall system accuracy is important for defining application areas of the system, and also for evaluating the cost - effectiveness of the system configuration. In practice, the images may have a sufficient resolution for identifying relative small objects such as fire hydrants. The positions can be determined at an accuracy of 30-50cm, for example. However, the positional errors may exceed the diameter of the fire hydrants. The important differences between resolution and accuracy should be distinguished. A relatively expensive hence important component in the system is INS. Systems developed for applications not requiring high-accurate spatial data may be able to use

12 alternatives such as gyroscopes or multiple GPS receivers to reduce the system costs. Application considerations Highway Application Because it employs dynamic data acquisition, mobile mapping technology can be directly used in highway related applications, such as traffic sign inventory, monitoring of speed and parking violations, generation of road network databases, and road surface condition inspection when laser technology is jointly applied. There are several advantages to using mobile mapping technology in highway applications. The data acquisition is performed without blocking the traffic assuming that traffic velocity is less than, for example, 70km/h. The information obtained is diverse - single collection can be used for multiple purposes. Moreover, since data can be both collected and processed in a short period of time, frequent and repetitive road surveys and database updating are both possible and affordable. Objects along roads and highways, for example traffic signs, light poles, bridges, road centerlines etc., are usually represented as clear image features in the image sequences. Therefore, they can be identified easily and measured interactively to build a spatial database. In order to extract a road centerline, an image sequence of the road is needed. Each image pair supplies a segment of the centerline. The successive road segments from the sequence are measured continuously and combined to produce the entire road centerline. Road centerlines and separating lines between lanes are painted in white or yellow. They have solid or dashed line patterns. Based on road surface and weather conditions, the quality of the lines in the image sequence varies. Manual extraction of the lines by the operator is relatively time-consuming, although it is superior to other traditional surveying methods. Automation of this procedure has been researched. From the image sequence, centerline features are enhanced and extracted automatically. The corresponding 3D centerline segments are then generated in the object space (He and Novak 1992). Another approach defines a 3D centerline model in the object space as a physical Snake model (Tao et al 1996). The Snake model is optimized to adjust the centerline shape using image features of the centerline as internal constraints, and geometric conditions derived from other sensors of the system (GPS and INS) as external constraints. This method for automatic centerline extraction and reconstruction is reliable for different road conditions and line patterns. On the other hand, road curb lines, as opposed to painted centerlines, are projected onto the images based on their geometric shapes and material types. Therefore, curb lines can be more difficult to extract and identify automatically. Currently, the

13 curb line databases are built using semiautomatic approaches. The system provides the user with projected curb lines in a stereo image pair. The user is asked to confirm if the line pair suggested by the system is correct. Sometimes, because of the image quality, the system cannot provide the suggestion. In this case, the operator is asked to digitize the curb segments manually. It is often required in road surface condition inspection that road surface cracks be located and measured. If the system is equipped with laser sensors, the depths between the sensors and the road surface are available as relative measurements. If the control data from GPS and INS are available, the depth data derived from the laser sensors can be integrated into the global reference system and used to generate a Digital Road Surface Model (DRSM). The DRSM provides a geometric description of the road surface. This can be used to detect the locations, sizes, and shapes of the road surface cracks. If the digital image sequence of the same road is processed and georeferenced, each grid point of the DRSM can be assigned with a gray scale from the corresponding pixel of the image sequence. In this way the images appear to be draped onto the DRSM and a 3D road surface image is generated. By observing this 3D image using 3D visualization tools, road surface conditions including cracks can be illustrated more efficiently. On the other hand, the DRSM can also be built from the stereo image sequence along the road. Corresponding road surface points appearing in a stereo pair can be measured by digital image matching techniques. Since one point may appear in more than one successive stereo pair, a point covered by an obstacle, such as a vehicle, in one stereo pair may be visible in the preceding or subsequent pairs. A gridding procedure is applied to calculate grid points of the DRSM. In comparison to the digital matching method using stereo image sequences, laser sensors generate more reliable surface models, because the depths are always measured directly. However, in the model built by the matching based method, there may be areas with points that are not measured, but interpolated, because the areas are invisible or do not have sufficient image texture for image matching. Application to Facility Mapping Another useful application of mobile mapping technology is in the area of facility mapping. High voltage power transmission lines can be photographed by the mobile mapping system and their positions can be measured from the image sequences. There are a number of important parameters which can be calculated from the mobile mapping data, for instance the positions of the poles and/or towers; the positions of the insulators on each line which support the suspending transmission line segments; and the lowest points of the suspending line segments. In order to capture these desired transmission line features, the cameras have to be oriented somewhat upwards to aim at the towers and line segments. Consequently, a large portion of the resulting images contains the sky as the background. This makes it easy to distinguish the targets from the background in the images because of the high contrast between them. Based on the same reason, automatic extraction of the transmission line segments in the image space is also possible.

14 However, if epipolar geometry (Moffitt and Mikhail 1980, Li 1996) is used to determine the line segments in the object space, the 3D points along the line segments to be measured are dependent on the intersections between the epipolar lines and the line segments in the images (Figure 4). In case I, two cameras are forward looking, with an upper-left angle. For the k-th image pair, the epipolar line formed by Imageleft,k and Imageright,k are almost parallel to the transmission lines. The intersections thus obtained are of low accuracy considering the effect of both the intersection accuracy and errors of the epipolar lines caused by imperfect orientation parameters. This will consequently affect the accuracy of the 3D line segments in the object space derived thereby. There are three options for solving this problem: a) In case I of Figure 4, if an appropriate overlapping area is available, subsequent images such as Imageleft,k and Imageleft,k+1 are used to form a stereo pair so that the Epipolar-line(left,k)(left,k+1) has a better intersecting angle with the transmission line. b) In case II of Figure 4, the cameras are oriented toward the left side. This will also result in effective intersecting angles. c) A hardware-based stereo viewing system can be used, for example using a polarized system with a stereo glass and a special monitor. In this case, a 3D stereo model can be reconstructed. The operator is then able to view the transmission line in 3D and measure points along the line three-dimensionally. d) Better results can also be achieved by combining data acquired by laser ranging by a helicopter if available (Krabill 1989). Some objects such as fire-hydrants and manholes have symmetric geometric shapes and are less dependent on the camera positions and orientations. Thus, they appear similar in images. Based on the geometric shapes of the objects, simulated lighting sources, and material characteristics, artificial images can be generated. These artificial images can then be compared with the image features in the sequences. A matching procedure between the artificial images and real images is performed. In this way, both the geometric information and attributes of fire-hydrants and manholes appearing in the image sequences can be extracted efficiently. A perspective on mobile mapping One of the criteria used to measure the quality of the system is the accuracy of the location of objects measured using the acquired data. This accuracy is strongly influenced by the control data collected by the navigation sensors and the quality of the mapping sensors. The components of mobile mapping systems function efficiently and reliably as individual and independent systems in various applications. A mobile mapping system requires that these components work cooperatively. Another factor to consider is cost. A highly accurate strapdown INS is currently the most costly component in the system. Low level INS and gyroscopes may be used, but they do not supply the same quality angular parameters which can be employed, for example, in camera orientation.

15 The high cost of the INS make the entire system relatively expensive. If high accuracy is not essential, low cost gyroscopes can be used in order to make the system more affordable. The image resolution of the cameras has a great influence on the accuracy of the photogrammetric intersections of object points. This is especially critical because the physical baseline of the cameras is limited by the dimension of the land based vehicle. For an object far from the cameras, the intersecting angle is small, and one pixel error in the image will result in to a large error along the track. High resolution cameras up to 4096pixels x 4096pixels are available but are rather expensive. In addition, the images acquired by high resolution cameras occupy a large memory and storage space. If multiple high resolution color cameras are employed, the problems of efficient image data transmission and medium storage during the data acquisition, efficient object measurement and attribute extraction, and data archiving will be addressed. GPS should provide "continuous" and consistent data for absolute positional control. However, there are cases where GPS signals are blocked by high-rise buildings, tunnels, and other objects. An integration of GPS signals at the last point before the signal blocking and the first point of signal recovery with INS trajectories bridges the gap of the control data. On the other hand, there are situations in which GPS signals are only blocked for a very short period, affecting one or two exposure stations. If not detected and corrected, these errors will distort both the positions of the camera exposure centers, and the object points derived from the image sequences. Therefore, an automated systematic quality checking procedure should be implemented to examine the GPS and INS data. If such an inconsistency exists, positions of a point derived from different image pairs of different exposure stations will show a large difference. Otherwise, they should have the same position within a certain tolerance. This quality checking procedure guarantees the quality of the control data used for camera orientation. To overcome the difficulties and to improve the mobile mapping technology, the following further research and development should be conducted: If a secondary local navigation network were available in areas where GPS signals are blocked, vehicle positioning and sensor control would be more reliable anytime and anywhere. Further improvement in downloading the mapping data acquired from the vehicle by using wireless communication technology should be explored. Larger format CCD chips should be employed to increase the resolution of the images and consequently, the accuracy of the photogrammetrically intersected object points. This is especially important if a similar configuration is used on helicopters or aircrafts where one pixel represents a much larger area on the ground.

16 More efficient image processing and sequential estimation algorithms should be researched and developed in order to make a good use of the large amount of high resolution data and characteristics of sequential images. Enhancement of the automation of object recognition and attribute extraction would improve the efficiency of GIS database generation from the georeferenced image sequences. This would also contribute to the reduction of the significant difference between the speed of mobile mapping data acquisition and that of the subsequent data processing. An alternative way to bridge a period of GPS outage using INS is to perform a terrestrial photogrammetric triangulation. A strip of overlapping photos are relatively oriented to form a strip model covering the GPS gap. The GPS data available at the two ends of the strip can be used to orient the strip model absolutely. Automation of this labor-intensive and timeconsuming procedure is deemed necessary. Important issues involved in the automation include automatic tie point selection, conjugate tie point searching, and absolute strip orientation using GPS data. Ideally, depending on the degree of the automation, the terrestrial photogrammetric triangulation may be performed using all images. Efforts in mobile mapping research and development have been made by researchers and engineers since last decade. The technology has evolved to such a degree as to allow mapping and GIS industries to use it for high flexibility in data acquisition. More information can be gained in less time, and with less effort, while achieving high productivity. In addition, a successful extension of this technology to helicopter - borne and airborne (Bossler 1996) systems will provide us with a powerful tool for small scale and medium scale spatial data acquisition and database updating. Acknowledgment Research support from the Natural Sciences and Engineering Research Council of Canada (NSERC) is acknowledged. Reviewers comments are appreciated.

17 References Ackermann, F. and H. Schade Application of GPS for Aerial Triangulation. PE&RS, Vol.59, No.11, pp Bossler, J.D Airborne Integrated Mapping System. GIM, July, 1996, pp Bossler, J.D., C.C. Goad, P.C. Johnson, and K. Novak GPS and GIS - Map the Nation's Highways. GeoInfo Systems, March 1991, pp Bossler, J.D. and C.K. Toth Feature Positioning Accuracy in Mobile Mapping: Results Obtained by the GPSVanTM. Int. Archives of Photogrammetry and Remote Sensing, Vol.XXXI, part B2, pp Cannon, M.E The Use of GPS for GIS Georeferencing: Status and Applications. Proceedings of the Canadian Conference on GIS, pp Cavanaugh, D.B Calibration of the APALS Database. Proceedings of 1995 Mobile Mapping Symposium, ASPRS, pp.1-8. Chapman, M. and L. Baker High-Speed Video Road Survey in Singapore. GIS Asia Pacific, April, 1996, pp Colomina, I Combined Adjustment of Photogrammetric and GPS Data. 42nd Photogrammetric Week, Stuttgart, Germany, pp Cosandier, D. and M.A. Chapman High Precision Target Location for Industrial Metrology. SPIE Vol.1820, Videometrics, pp Cosandier, D., M.A. Chapman and T. Ivanco Low Cost Attitude Systems for Airborne Remote Sensing and Photogrammetry. The Canadian Conference on GIS, Ottawa, March Da, R. and G. Dedes Nonlinear Smoothing of Dead Reckoning Data with GPS Measurements. Proceedings of 1995 Mobile Mapping Symposium, ASPRS, pp El-Sheimy, N A Mobile Multi-Sensor System for GIS Applications in Urban Centers. Int. Archives of Photogrammetry and Remote Sensing, Vol.XXXI, part B2, pp El-Sheimy, N. and K.-P. Schwarz Kinematic Positioning in Three Dimension Using CCD Technology. Proceedings of IEEE - Vehicle Navigation & Information Systems (VNIS) pp Gruen, A. and T.P. Kersten Sequential Estimation in Robot Vision. Photogrammetric Engineering and Remote Sensing, Vol.61, No.1, pp

18 He, G. and K. Novak Automatic Analysis of Highway Features from Digital Stereo-Images. Int. Archives of Photogrammetry and Remote Sensing, Vol.XXIX, part B3, pp He, G Design of a Mobile Mapping System for GIS Data Collection. Int. Archives of Photogrammetry and Remote Sensing, Vol.XXXI, part B2, pp Hein, G.W Precise Kinematic GPS/INS Positioning: A Discussion on the Applications in Aerophotogrammetry. 42nd Photogrammetric Week, Stuttgart, Germany, pp Hock, C., W. Caspary, H. Heister, J. Klemm, and H. Sternberg Architecture and Design of the Kinematic Survey System KiSS. Proceedings of the 3rd Int. Workshop on High Precision Navigation, Stuttgart, April, 1995, pp Jacobsen, K Correction of GPS Antenna Position for Combined Block Adjustment. Proceedings of ASPRS/ACSM, pp Kimura S., H. Kano, and T. Kanade CMU Video - Rate Stereo Machine. Proceedings of 1995 Mobile Mapping Symposium, ASPRS, pp Krabill, W.B GPS Applications to Laser Profiling and Laser Scanning for Digital Terrain Models. 42nd Photogrammetric Week, Stuttgart, Germany, pp Krakiwsky, E.J. and R.L. French Japan in the Driver's Seat. GPS World, October 1995, pp Leick, A GPS Satellite Surveying. Second Edition, John Wiley & Sons, Inc., New York. Lapine, L.A Analytical Calibration of the Airborne Photogrammetric System Using A Priori Knowledge of the Exposure Station Obtained by Kinematic GPS Techniques. Report No.411, 1991, Department of Geodetic Science and Surveying, The Ohio State University. Li, R Building Octree Representations of 3D Objects in CAD/CAM by Digital Image Matching Techniques. PE&RS Vol.58, No.12, pp Li, R Design and Implementation of a Photogrammetric Geo-Calculator in a Windows Environment. PE&RS Vol.62, No.1, pp Li, R., K.-P. Schwarz, M.A. Chapman and M. Gravel Integrated GPS and Related Technologies for Rapid Data Acquisition. GIS World, April 1994, pp Li, R. M.A. Chapman, L. Qian, Y. Xin and K.-P. Schwarz Rapid GIS Database Generation Using GPS/INS controlled CCD cameras. Proceedings of the Canadian Conference on GIS, pp