INTEGRATION OF LASER AND PHOTOGRAMMETRIC DATA FOR CALIBRATION PURPOSES

Transcription

1 INEGION OF LE ND PHOOGMMEIC D FO CLIION PUPOE. F. Habib a M.. Ghanma a M. F. Morgan a E. Mitishita b a Department of Geomatics Engineering Universit of Calgar 500 Universit Drive NW Calgar N N4 Canada habib mghanma@geomatics.ucalgar.ca mfmorgan@ucalgar.ca b Departamento de Geomática Universidade Federal Do Paraná Caia Postal Curitiba Paraná rasil mitishita@ufpr.br P: WG I/5 Platform and ensor Integration KE WOD: Laser scanning egistration Calibration Fusion Feature Modelling C: Laser scanners are becoming popular since the provide fast and dense geometric surface information. However sudden elevation changes along the surface are not clearl visible in the laser data due to the sparse distribution of captured points. In general laser data provides high densit surface information in homogenous areas and low densit surface information elsewhere i.e. object space break-lines. Photogrammetr on the other hand provides less dense surface information but with high qualit especiall along object space discontinuities. Hence a natural snerg of both sstems can be inferred and consequentl integration of the respective data would lead to higher qualit surface information than that obtained through the use of a single sensor. However prior to such integration both sstems should be precisel calibrated and aligned. he calibration is usuall carried-out for each sstem independentl using additional control information. In this paper the calibration of the laser and photogrammetric sstems is evaluated b checking the qualit of fit between co-registered photogrammetric and laser surfaces. he paper starts b introducing a registration procedure where a set of linear features is etracted from both sets. First planar surfaces from laser data are etracted and adjacent planes are intersected to determine three-dimensional straight line segments. econdl linear features from the photogrammetric dataset are obtained through aerial triangulation. mathematical model for epressing the necessar constraints for the alignment of conjugate photogrammetric and laser straight lines is established. he model ensures that corresponding straight lines will be collinear after registering the two datasets relative to a common reference frame. he qualit of fit between the registered surfaces is then used to evaluate and/or improve the calibration parameters of the photogrammetric and laser sstems. In this paper an eperiment with real data is used to illustrate this concept. he registered surfaces in this eample revealed the presence of sstematic inconsistencies between the photogrammetric and laser sstems. he pattern of these inconsistencies is found to resemble the effect of un-calibrated lens distortion. In this case the laser data is used as control information for the determination of lens distortion which when considered leads to a better fit between the registered surfaces. he estimated lens distortion using the laser was found to be ver close to that determined through a rigorous camera calibration procedure.. INODUCION Laser scanners are becoming an increasingl accepted tool for acquiring 3D point clouds that represent scanned objects with millimetre precision. s can be inferred from the name laser scanning is a non-contact range measurement based on emitting a laser light pulse and instantaneousl detecting the reflected signal. his should be coupled with high-qualit GP/IN units for tracking the position and orientation of the range finder as it scans over objects and surfaces under consideration. he sparse and positional nature of laser data makes it ideal for mapping homogeneous surfaces but lacks the abilit to capture objects break-lines with reliable qualit. nother drawback is that laser data has no inherent redundanc and its planimetric accurac is worse than the vertical Maas H.-G. 00 in addition to little or no semantic information. However the continuous development of laser sstems in the aspect of reduced hardware sie and increased resolution and densit makes it an increasingl favoured option in a variet of applications especiall where rapid and accurate data collection on phsical surface is required chenk and Csathó 00. On the other side photogrammetric data is characteried b high redundanc through observing the desired features in multiple images making it more suited for mapping heavil populated areas. ichness in semantic information and dense positional information along object space break lines add to its advantages. Nonetheless photogrammetr has its own drawbacks; where there is almost no positional information along homogeneous surfaces and vertical accurac is worse than the planimetric accurac. major eisting obstacle in the wa of automation in photogrammetr is the complicated and sometimes unreliable matching procedures especiall when dealing with large scale imager over urban areas. It can be clearl observed that both photogrammetr and laser data have unique characteristics that make them preferable in certain applications. One can notice that a disadvantage in one technolog is contrasted b an opposite strength in the other. Hence integrating the two sstems would lead to higher qualit surface information altsavias 999. However the complementar information can be full utilied onl after precise calibration of both sstems which is separatel implemented for each sstem. he snerg would be considered complete after aligning the photogrammetric and laser data models relative to a common reference frame. Habib and chenk 999; Postolov et al his paper introduces a registration procedure through which the calibration of photogrammetric and laser scanning sstems is assessed. he suggested technique emphasies the tpe and

2 etraction of registration primitives in addition to the registration steps required to reveal an calibration discrepancies in the sstems. Most registration methodologies use discrete points as the sole primitive for solving the registration problem between two datasets. uch methodologies are not applicable to laser scanned surfaces since the correspond to laser footprints instead of distinct points that could be identified in imager altsavias 999. Conventionall surface-to-surface registration and comparison have been achieved b interpolating both datasets into a uniform grid. he comparison is then reduced to estimating the necessar shifts b analing the elevations at corresponding grid posts Ebner and Ohlhof 994; Kilian et al everal issues can arise with this approach. First the interpolation to a grid will introduce errors especiall when dealing with captured surfaces over urban areas. Moreover minimiing the differences between the surfaces along the -direction is onl valid when dealing with horiontal planar surfaces Habib and chenk 999. Postolov et al. 999 presented another approach which works on the original scattered data without prior interpolation. However the implementation procedure involves an interpolation of one surface at the location of conjugate points on the other surface. dditionall the registration is based on minimiing the differences between the two surfaces along the -direction. chenk 999 introduced an alternative approach where distances between points of one surface along surface normals to locall interpolated patches of the other surface are minimied. Habib and chenk 999 and Habib et al. 00 implemented this methodolog within a comprehensive automatic registration procedure. uch an approach is based on processing the photogrammetric data to produce object space planar patches. his might not be alwas possible since photogrammetric surfaces provide accurate information along object space discontinuities while suppling almost no information along homogeneous surfaces with uniform teture. In this paper the registration procedure will utilie straight line primitives and 3D similarit transformation for aligning the photogrammetric model relative to the laser data reference frame. he following section previews the components of the general registration paradigm and the particulars of appling each component to the photogrammetric and laser datasets under consideration. he last two sections cover the eperimental results as well as the conclusions and recommendations for future work. straight line features are selected for this purpose. his choice is motivated b the fact that such primitives can be reliabl accuratel and automaticall etracted from photogrammetric and laser datasets. he procedure adopted to etract straight lines from the photogrammetric and laser datasets and how the are included in the overall alignment procedure is described below. Photogrammetric straight line features: he representation scheme of 3D straight lines in the object and image space is central to the methodolog for producing such features from photogrammetric datasets. epresenting object space straight lines using two points along the line is the most convenient representation from a photogrammetric point of view since it ields well-defined line segments Habib et al. 00. On the other hand image space lines will be represented b a sequence of -D coordinates of intermediate points along the feature. his appealing representation can handle image space linear features in the presence of distortions as the will cause deviations from straightness. Furthermore it will allow for the inclusion of linear features in scenes captured b line cameras since perturbations in the flight trajector would lead to deviations from straightness in image space linear features corresponding to object space straight lines Habib et al. 00. Manipulating tie straight lines appearing in a group of overlapping images begins b identifing two points in one Figure a or two images Figure b along the line under consideration. hese points are then used to define the corresponding object space line segment. It is worth mentioning that these points need not be identifiable or even visible in other images. Intermediate points along the line are measured in all overlapping images. imilar to the end points the intermediate points need not be conjugate Figure.. MEHODOLOG he registration process aims at combining multiple datasets acquired b different sensors in order to reach better accurac and enhanced inference about the environment than could be attained through using onl one sensor. he following subsections address the components and issues necessar for an effective registration paradigm rown 99.. egistration primitives o register an two datasets certain common features have to be identified and etracted from both sets. uch features will subsequentl be used as the registration primitives relating the datasets together. he tpe of chosen primitives greatl influences subsequent registration steps. Hence it is crucial to first decide upon the primitives to be used for establishing the transformation between the datasets in question. In this paper Figure. End points defining the object line are either measured in one image a or two images b. he relationship between the image coordinates of the line end points { } and the corresponding ground coordinates { } is established through the collinearit equations. Hence four equations are written for each line. he intermediate points are included into the adjustment procedure through a mathematical constraint which states that the vector from the perspective centre to an intermediate image point along the line is contained within the plane defined b the perspective centre of that image and the two points defining the straight line in the object space Figure. hat is to sa for a given intermediate point a

3 constraint that indicates the points { Oi Oi Oi and i i 0} are coplanar is introduced and mathematicall described b Equation. V V V 0 3 In the above equation V is the vector connecting the perspective centre to the first end point along the object space line V is the vector connecting the perspective centre to the second end point along the object space line and V is the 3 vector connecting the perspective centre to an intermediate point along the corresponding image line. It is important to note that the three vectors should be represented relative to a common coordinate sstem e.g. the ground coordinate sstem. he constraint in Equation incorporates the image coordinates of the intermediate point the Eterior Orientation Parameters EOP the Interior Orientation Parameters IOP including distortion parameters as well as the ground coordinates of the points defining the object space line. uch a constraint does not introduce an new parameters and can be written for all intermediate points along the line in the imager. he number of constraints is equal to the number of intermediate points measured along the image line. In this paper laser straight line features will be used as a source of control to align the photogrammetric model. o etract such lines suspected planar patches in the laser dataset are manuall identified with the help of corresponding optical imager Figure 3. he selected patches are then checked using a least-squares adjustment to determine whether the are planar or not and to remove blunders. Finall neighbouring planar patches with different orientation are intersected to determine the end points along object space discontinuities between the patches under consideration. he datum for the laser lines is directl established b a combination of high-qualit GP/IN units installed onboard of the sensor platform. a Figure 3. Manuall identified planar patches in the laser data a guided b the corresponding optical image b. b. egistration transformation function Figure. Perspective transformation between image and object space straight lines and the coplanarit constraint for intermediate points along the line. In some applications photogrammetric lines are used as control lines instead of being regular tie lines. In this situation the object coordinates of line end points are known hence these points need not be measured in an of the images. Consequentl image space linear features are represented onl b a group of intermediate points measured in all images. fter the identification and etraction of straight lines from imager a photogrammetric model is generated through a photogrammetric triangulation using an arbitrar datum without an control information. his arbitrar datum is defined b fiing seven coordinates of an three well-distributed points. Laser straight line features: he increasing recognition of laser scanning as a favourable data acquisition tool b the photogrammetric communit led to a number of studies aiming at pre-processing laser data. he major goal of such studies ranges from simple primitive detection and etraction to more complicated tasks such as segmentation and perceptual organiation Csathó et al. 999; Lee and chenk 00; Filin 00. t this point a photogrammetric model is generated from the photogrammetric triangulation using an arbitrar datum without knowledge of an control information. In addition a set of conjugate photogrammetric-laser lines has been manuall identified. hese lines in both datasets are identified b their end points. It is important to reiterate that the end points of such conjugate lines are not required to be conjugate. n essential propert of an registration technique is the tpe of transformation or mapping function adopted to properl overla the two datasets. In this paper a 3D similarit transformation is used as the registration transformation function Equation. uch transformation assumes the absence of sstematic biases in both photogrammetric and LID surfaces Filin 00. However the qualit of fit between conjugate primitives can be analed to investigate the presence of such behaviour. where: a a a is the scale factor is the translation vector between the origins of the photogrammetric and laser data coordinate sstems is the 3D orthogonal rotation matri between the two coordinate sstems a a a are the photogrammetric point coordinates and are the coordinates of the corresponding point relative to the laser data reference frame..3 imilarit measure he role of the similarit measure is to mathematicall epress the relationship between the attributes of conjugate primitives in overlapping surfaces. he similarit measure formulation depends on the selected registration primitives and their respective attributes as well as the transformation function. In

4 this paper the similarit measure formulation has been incorporated in mathematical constraints ensuring the coincidence of conjugate linear features after establishing the proper co-registration between involved surfaces. eferring to Figure 4 the two points describing the line segment from the photogrammetric model undergo a 3D similarit transformation onto the line segment from the laser dataset. he objective here is to introduce the necessar constraints to describe the fact that the model segment coincides with the object segment after appling the absolute orientation transformation function. Figure 4. imilarit measure between photogrammetric and laser linear features. For the photogrammetric point this constraint can be mathematicall described as in Equation 3. λ 3 Equation 4 shows the constraint for point. λ 4 where λ and λ are scale factors. ubtracting Equation 4 from Equation 3 ields: λ λ 5 Dividing both parts of Equation 5 b λ -λ and substituting λ for /λ -λ Equation 6 is produced. Equation 6 emphasies the concept that model line segments should be parallel to the object line segments after appling the rotation matri. o recover the elements of the rotation matri Equation 6 is further manipulated and rearranged b dividing the first and second rows b the third to eliminate λ Equations 7. λ pair of conjugate line segments ields two equations which contribute towards the estimation of two rotation angles the aimuth and pitch along the line. On the other hand the roll angle across the line cannot be estimated due to singularities. Hence a minimum of two non-parallel lines is needed to recover the three elements of the rotation matri Figure 5. Figure 5. ingular and optimum configuration to recover rotation angles o determine the scale factor and the shift components appl the rotation matri to the coordinates of the first point defining the photogrammetric line which ields Equation 8. λ 8 where [ ] [ ] earranging the terms ields: λ 9 In Equation 9 eliminate λ b dividing the first and second rows b the third Equations 0. he same applies to point and Equations can be written. 0 Combining Equations 0 and produces the two independent constraints as shown in Equations. Equations can be written for each line in one dataset and its conjugate in the other. Consequentl two pairs of line segments ielding four equations are required to solve for the four unknowns. If the lines were intersecting the shift components can be estimated using the intersection points but the scale factor cannot be recovered. s a result at least two noncoplanar line segments are needed to recover these parameters Figure 6. In summar a minimum of two non-coplanar line segments is needed to recover the seven elements of the 3D similarit transformation. 3D imilarit ransformation Model LID

5 Planar surfaces are then fitted through the selected patches from which neighbouring planar surfaces are intersected to produce object space line segments. total of twent-three well distributed 3D edges within the area of interest have been identified along ten buildings from three laser strips. Least-squares adjustment is then used to solve for the parameters of the 3D similarit transformation function and the results are shown in able. visual presentation of datasets after transformation is shown in Figure 8. Figure 6. ingular and optimum configuration to recover scale and shift components t this point the components of the registration paradigm have been addressed. traight line segments are chosen as the registration primitives along with a 3D similarit transformation function. lso the similarit measure is formulated based on the selected primitives and transformation function. he qualit of fit represented b the resulting variance component from the similarit measure as well as the residuals and discrepanc between conjugate features will be used to validate and check the qualit of the calibration parameters associated with the imaging and ranging sstems. cale ± m 4.8 ±0.74 m -.4 ±0.45 m ± ± ± ± able. 3D similarit parameters between laser and photogrammetr models. 3. EPEIMENL EUL In order to verif the methodolog and procedure imager and laser data over an urban area were collected Figure 7. Figure 8. erial photogrammetric and laser datasets after transformation Figure 7. erial image of area under consideration CNON EO D digital camera piel sie.5µm focal length mm was used to capture twent-three overlapping images in three flight lines. ased on a fling height of 00 m a base of 70 m and assuming a one piel measurement error the epected planimetric accurac is estimated as 0.09 m while the vertical accurac is epected to be 0.36 m with an overall spatial accurac of 0.37 m. From the laser scanner hardware and mission specifications the spatial laser data accurac is epected to be in the range of 0.35 m. With the above anticipated accuracies the surfaces are epected to have a discrepanc in the range of 0.5 m. traight line segments as well as some tie points were measured as described earlier and then incorporated in a bundle adjustment procedure with an arbitrar datum. he output of the bundle adjustment included the ground coordinates of tie points in addition to the ground coordinates of points defining the object space line segments. In the laser data homogeneous patches have been manuall identified to correspond to that of selected features in imager. o assess the qualit of fit the mean normal distance between the laser and transformed photogrammetric line segments turned out to be 3.7 m a surprisingl poor result considering the camera flight mission and laser scanner specifications. he epected surface fit was in the range of a sub-meter. closer look at the side view in Figure 8 the discrepanc revealed that the pattern of deviation between the laser and photogrammetric features is similar to deformations arising from ignored radial lens distortion.o determine the radial lens distortion of the implemented camera two alternatives were followed. he first alternative implemented the laser features as control information within the bundle adjustment procedure in a self-calibration mode allowing for the derivation of an estimate for the radial lens distortion. he estimated radial lens distortion coefficient turned out to be mm -. he second alternative determined an estimate of the radial lens distortion through a bundle adjustment with self-calibration involving imager captured from a test field with numerous control points which had been surveed earlier. he estimated radial lens distortion coefficient turned out to be mm - which is almost identical to the value determined b implementing the laser features as control within the photogrammetric bundle adjustment. fterwards the registration procedure had been

6 repeated after considering the radial lens distortion. he new parameters of the transformation function are presented in able. fter considering the radial lens distortion the mean normal distance between the laser and transformed photogrammetric line segments turned out to be 0.58 m which is within the epected accurac range. sharp drop in the standard deviations of the transformation function parameters also took place as can be seen when comparing ables and. he overall improvement in the spatial discrepancies after introducing the radial lens distortion verifies its eistence. cale.0803 ± m 7.05 ±0.8 m.4 ±0.3 m -4.7 ± ± ± ± able. 3D similarit parameters between laser and photogrammetr models after distortion compensation. 4. CONCLUION ND ECOMMENDION naling the previous results a set of conclusions can be etracted from this stud mainl; the efficienc of the suggested registration procedure in identifing the sstematic discrepancies between the involved surfaces. fter a closer look at the discrepancies behaviour it was possible to justif the cause and take the necessar remedial measures to remove such errors. lso straight line features proved its suitabilit to establish a common reference frame for the laser and photogrammetric surfaces a result that has been suggested b prior research work. he involved datasets in the eperimental section illustrated the compatibilit between laser and photogrammetric surfaces. However it is important to precisel calibrate both sstems to guarantee the absence of sstematic biases. In addition the two surfaces must be relative to the same reference frame as a prerequisite for an further integration between the two datasets. For eample optical imager can be rendered onto the laser data to provide a realistic 3D tetured model of the area of interest. Further research is required to address the automatic etraction of different tpes of primitives from the surfaces in question. Developing an automatic matching strateg between laserderived and photogrammetric features is an interesting etension. For eample Modified Iterated Hough ransform MIH can be used to simultaneousl determine the correspondence between conjugate primitives in overlapping surfaces and the parameters involved in the registration transformation function. he tpe of transformation function will also be looked at. o far 3D similarit transformation has been assumed as the registration transformation function relating overlapping surfaces. Future work will investigate the discrepanc pattern for different errors and factors such as GP/IN/Laser spatial and rotational biases or biases in the IOP of the involved cameras. 5. EFEENCE altsavias E comparison between photogrammetr and laser scanning IP Journal of Photogrammetr & emote ensing 54: rown L. G. 99. surve of image registration techniques CM computing surves 44: Csathó. M. K. oer and. Filin 999. egmentation of laser surfaces International rchives of Photogrammetr and emote ensing 33W4: Ebner H. and. Ohlhof 994. Utiliation of ground control points for image orientation without point identification in image space International rchives of Photogrammetr and emote ensing 303/:06. Filin. 00. urface clustering from airborne laser scanning data International rchives of Photogrammetr and emote ensing 33:9-4. Habib. and. chenk 999. New approach for matching surfaces from laser scanners and optical sensors International rchives of Photogrammetr and emote ensing 33W4:55-6. Habib.. Lee and M. Morgan 00. urface matching and change detection using the modified Hough transform for robust parameter estimation Photogrammetric ecord Journal 798: Habib.. Lee and M. Morgan 00. undle adjustment with self-calibration using straight lines Photogrammetric ecord Journal 700: Kilian J. N. Haala and M. Englich 996. Capture and evaluation of airborne laser scanner data International rchives of Photogrammetr and emote ensing 33: Lee I. and. chenk 00. 3D perceptual organiation of laser altimetr data International rchives of Photogrammetr and emote ensing 343W4: Maas H.-G. 00. Methods for measuring height and planimetr discrepancies in airborne laserscanner data Photogrammetric Engineering and emote ensing 689: Postolov.. Krupnik and K. McIntosh 999. egistration of airborne laser data to surfaces generated b photogrammetric means International rchives of Photogrammetr and emote ensing 33W4: chenk Determining transformation parameters between surfaces without identical points echnical eport Photogrammetr No. 5 Department of Civil and Environmental Engineering and Geodetic cience OU pages. chenk. and. Csathó 00. Fusion of LID data and aerial imager for a more complete surface description International rchives of Photogrammetr and emote ensing 343: CKNOWLEDGEMEN he authors would like to thanks Mosaic Mapping Inc for suppling the aerial and laser datasets on which the eperimental work was conducted.