A New Approach for the Mathematical Modeling of the GPS/INS Supported Phototriangulation using Kalman Filtering Process

Transcription

1 A New Approach for the Mathematical Modeling of the GPS/INS Supported Phototriangulation using Kalman Filtering Process A. Azizi, M.A. Sharifi and J. Amiri Parian Surveying and Geomatics Engineering Faculty of Engineering University of Tehran Tehran, IRAN. Biography Ali Azizi holds a Ph.D. from the University of Glasgow, Great Britain. He has been an assistant professor in the Faculty of Engineering, University of Tehran, since His research interests are the real time close range photogammetry for deformation monitoring, medical photogrammetry and automatic feature extraction. Mohammad Ali Sharifi holds a B.Sc. in Civil and Surveying Engineering and an M.Sc. in Geodesy. He is currently an academic staff member of the Faculty of Engineering, University of Tehran. His research interests are: new optimization techniques as applied in geomatics, and GPS/INS solutions for structural deformation monitoring. Jafar Amiri Parian holds a B.Sc. in Civil and Surveying Engineering from the Faculty of Engineering University of Tehran and is currently a research student in photogrammetry working on three dimensional face reconstruction. Mr. Parian research interests are also in machine vision and non-parametric techniques for pattern recognition. Abstract In this paper a new approach for the mathematical modeling of the GPS/INS supported photo-triangulation based on the Kalman filtering process is proposed. The well known Kalman filtering is a linear recursive estimator based on the prediction and the measurement models and comprises five sequential process, namely: (1) state vector extrapolation, (2) covariance extrapolation, (3) Kalman gain computation, (4) state vector update and (5) covariance update. In the present project the requirements for the Kalman filtering process are satisfied by employing standard collinearity condition equations as the prediction model. The state vector in this problem is considered to include the coordinates of the projection centers of the camera during the exposure time, camera orientation parameters, and the ground coordinates of the tie points. The prediction model is accompanied by a measurement model that, in its simplest form, is considered to be a linear interpolation algorithm that accepts the GPS/INS observables during the aerial photography flight mission. Any additional observations such as the coordinates of the ground control points may also be tailored into the measurement model. These models are numerically updated by the periodic calculation of the Kalman gain factor that utilizes updated variance-covariance matrices of observation and prediction models. The whole process may be further extended to cover dynamically acquired digital images; i.e. those images acquired by the newly designed airborne three-line linear array CCD cameras. This paper explains the implemented mathematical model in some details. This is then followed by a report of the preliminary tests carried out on a simulated block of nearly 30 photographs. 1. Introduction Photo-triangulation started with the sequential model connection strategy to avoid the handling of large matrices, as the early generation of computers did not allow the computation of the large sparse matrices in an efficient manner. The sequential model connection requires a two-stage process, namely: strip/block formation and strip/block adjustment. The first stage engendered a very complicated error propagation across the strip and the block and hence error accumulation could be modeled only in an imperfect way assuming the accumulation of errors in a double summation manner. As a result, the use of patch-wise, global or conformal polynomial strip/block adjustment was inevitable. Due to the polynomial shortcomings regarding the number and distribution of control points and also taking into account the non-simultaneous adjustment character associated with the sequential method, this approach was considered to be far from ideal. The same holds true for the photobased sequential triangulation using 2-dimensional image coordinates measured with respect to the comparator coordinate system.

2 However, with the rapid improvements in the computer hardware both in terms of processor speed and memory capacity, the sequential approach was soon replaced by a more accurate simultaneous solution leading to the well known independent model and bundle photo-triangulation respectively. Again a considerable amount of research has been carried out on simultaneous block adjustment as well as the solution of large sparse matrices (see for example American Society of Photogrammetry, 1980). Until now the bundle adjustment is regarded to be the most accurate and effective way of photo-triangulation especially when it utilizes the appropriate self calibration parameters (see Ebenr, 1975). The auxiliary data (GPS/INS) was also incorporated successfully and effectively in the simultaneous bundle or independent model adjustments (Ackerman, 1992) Nevertheless, taking into consideration the potential power of signal processing inherent in the random process filtering techniques, the classical sequential solution can be regarded again as an alternative solution to the simultaneous approach. In particular when different sources of observations are available (e.g. GPS/INS, ground control points, etc.), the efficient Kalman filtering, provided that the condition required by it is fulfilled, may be incorporated to the sequential strategy to give the following advantages: (a) error accumulation are avoided, provided that the independent observations for the projection centers are available. (b) in the case of inaccurate observations (i.e. those observations contaminated by high level of random noise), a reasonable solution for the strip or the block may still be obtained by Kalman filtering under certain conditions which will be discussed in section 2. (c) the solution of large matrices for large blocks of photos, can be avoided without the loss of accuracy. (d) sequential adjustments are performed as the operator measures and transfers the tie points from one photo to the next. This means that a sequential improvements also occurs by the Kalman gain as will be discussed in section 3. That is, as the operator approaches the last series of photos, the final results are also obtained. The sections that follow formulate the basic idea. This will be followed by a report of the early experiments conducted on simulated data. 2. The Kalman filtering model To grasp the concept of the Kalman filtering, let us start with the Wiener filter as applied in the field of image processing. Consider a linear shift invariant (LSI) system having h(x,y) as its point spread function, the relationship between the input f(x,y) and the output g(x,y) of this system can be expressed by the following convolution operation, g ( x, y ) = f ( x, y )* h( x, y ) (1) In the absence of random noise, a solution for f(x,y) can be easily obtained by an inverse solution of Eq. 1 which in space domain leads to the iversion of a block circulant matrix, H, constructed from h(x,y). The solution is greatrly simplified by diagonalization of H which in the final analysis is equivalent to the solution of Eq.1 in frequency domain, G (U,V ) = F(U,V ).H(U,V ) (2) where, F(U,V), G(U,V) and H(U,V) are the Fourier transforms of f(x,y), g(x,y) and h(x,y) respectively. The inverse solution of Eq. 2 yields an estimated value for the input spectrum Fˆ (U,V), G(U,V ) Fˆ (U,V ) = (3) H(U,V ) The inverse Fourier transform of Eq. 3 gives the estimated value, fˆ ( x, y ) of the input function in space domain. However, this solution deteriorates drastically as the signal to noise ratio approaches the unity. In this case the neglected term n(x,y), i.e. the random noise, must be explicitly applied to Eq. 2, i.e., g(x,y) = f(x,y) * h(x,y) + n(x,y) (4) in Eq. 4, n(x,y) is unknown, and hence, this equation cannot be solved for both n(x,y) and f(x,y). However, if n(x,y) is random with normal distribution, (i.e. an ergodic random variable), then it can be demonstrated that autocorrelation function of n(x,y) is the same for all its membership functions. This means that, in the final analysis for the inverse solution of Eq. 4, a constraint based on the auto-correlation function of n(x,y) can be applied to the functional model. This constraint, which is non other than the Wiener filter, is given by, + Rgf (B)= w( α ). Rgg( β α ) dα (5) where, R gf is the cross correlation function of g(x,y) and f(x,y); R gg is the auto-correlation function of g(x,y); and w

3 is the Wiener filter that satisfies the condition expressed by Eq. 5. Fourier transforem of Eq. 5 leads to, P fg (U,V) = W(U,V). P gg (U,V) (6) thus, Pfg(U,V ) W(U,V ) = (7) Pgg(U,V ) where, W is the Fourier transform of w; P fg and P gg are the power spectral density functions of the input/output and the power spectrum of the output respectively. If either of g(x,y) or n(x,y) have zero mean value, Eq. 5 can be simplified as, Pff W(U,V ) = (8) ( Pff + Pnn ) In Eq. 8, P nn is the noise power spectrum. With respect to what is mentioned in the preceding paragraphs one can summarize the principle difference between the classical least squares solution and the Wiener filter as follows: In the least squares solution the mean square error which is minimized is defined as, MSE = + [ fˆ (x, y)- g(x, y)] 2 dxdy Where as, with the Wiener filter the minimized mean square error is defined according to, MSE = + [fˆ(x, y)- f(x, y)] 2 dxdy (9) (10) It is true that in Eq.10, f(x,y) is unknown, but as stated above, under certain conditions, its auto-correlation can be estimated. The above formulation is appropriate when dealing with LSI systems. Photo-triangulation dos not seem to lie within this category. Here the Kalman filtering intervenes through which the relationship between the input and the output can be set up by differential relations expressed in the form of matrix multiplication. This is the so called prediction model. The observation model is also needed to set the relations between the observed values and the predicted values. Now, having established these two models, Kalman gain, which is equivalent to the Wiener filter (Eq. 8) in so far as he definition for the MSE (Eq. 10) is concerned, can be calculated. Since in Kalman filtering, convolution in Eq. 5 is replaced by the matrix multiplication, one can conclude that autocorrelation functions in Eq. 5, have to be replaced by the covariance matrices. That is, R ff amd R gg, are replaced by P, the covariance matrices of predicted values and R, the covariance matrix of the observed values respectively. This makes the Kalman filter, K, as, P K = (11) (P + R ) The predicted values and the estimated covariance matrix for the predicted values are, then, updated using Kalman gain (Eq. 11) which in turn is evaluated by the updated values of the covariance matrix P. Thus, the Kalman filtering process are applied according to the following consecutive stages: (a) state vector extrapolation, i.e. calculation of the unknown parameters using the prediction model. (b) covariance extrapolation using error propagation model. (c) Kalman gain computation. (d) state up date. (e) covariance update. The above process are repeated recursively until the last data sets are processed. One can see that the Kalman filter is more flexible than the Wiener filter in so far as the inclusion of different sets of observations are concerned. Moreover, the sequential improvements associated with the Kalman filtering approach greatly reduces the matrix sizes of the prediction model. Having stated the general concept of Kalman process and its differences with the classical least squares solution, in the sections which follow, a description on the implementation of the above stated procedure for the photo-triangulation is presented. 3. GPS/INS supported sequential photo-triangulation using Kalman filtering process Regarding the great potentials of the Kalman filter in dealing with the sequential process, it is possible to use a simple resection/intersection strategy formulated for a pair of photographs. The linearized collinearity condition equations is given by, A ij.xj + Bij.Q i = Dij (12) where A ij is the matrix of partial derivatives of collinearity condition equations with respect to the exterior orientation parameters for the ith point in the jth photo; X is the matrix of the corrections to the initial values of j

4 the exterior orientation parameters; B ij is the matrix of partial derivatives with respect to the ground coordinates of tie points; Q i is the vector of corrections to the initial values of the ground coordinates of the tie points; and D ij is the first term of the Taylor s expansion. If GPS/INS observations are available, Eq.12 can be solved for adjacent photo pairs successively until the entire block of photos are traversed. Now returning to the Kaman filtering process, Eq. 12 can be regarded as the prediction model with Xj regarded as the state vector. The observation model may be stated as, Pj = g( Lj ) (13) where Lj is the observed values for the exterior orientation parameters obtained by the GPS/INS systems; and g expresses a kind of interpolation function correcting the time lapse associated with the time-lag registration of the GPS/INS observations and the instants of exposures. The interpolation function may be chosen to be any of the linear or the spline functions. Having set the prediction and the observation models, the Kalman gain can be calculated using the initial covariance matrices for these models. The initial predictions for the state vector and their covariance matrix is, then, updated by the Kalman gain. To express the sequence in more detail consider a block of two strips with 4 photos in each strip (see Fig.1). One of the following two options, according to the case, can be adapted: A- GPS/INS observations are available: (a) compute the initial values for the exterior orientation parameters. GPS/INS observations, corrected for the position and the time offsets, can be taken as initial values. (b) evaluate a covariance matrix, P, for the initial values and estimate a covariance matrix, R, for the observations. (c) calculate the Kalman gain (Eq. 11) using the covariance matrices computed in the previous stage. (d) apply the Kalman gain to update the predicted values, X k + 1, and their estimated covariance matrix, k 1, according to: P + Xk + 1 = Xk + Kk + 1( X 0 Xk ) (14) Pk + 1 = Pk Kk + 1( Pk) (15) where X k+1 ; P k+1 ; X o and K k+1 are the latest predicted/updated values of the exterior orientation parameters and their covariance matrix; the observed values for the exterior orientation parameters and the Kalman gain respectively. (e) solve the space resection/intersection for photos II and III using the updated exterior orientation parameters of photo II and the ground coordinates of the tie points: 2,4 and 6, that were estimated in the first stage. The space resection/intersection, in this stage, is written for the tie points: 2,4,6,7,8 and 9, taking the predicted/updated values of the points: 2,4,6 and the orientation parameters of photo II, as known values. The solution of the equations yields estimated values for the orientation elements of photo III as well as the ground coordinates for the tie points 7,8 and 9. The estimated orientation elements and their covariance matrices are subsequently updated by the Kalman gain. (f) repeat the procedures from the first stage, solving the space resection/intersection for photos III and IV. This time taking the predicted/updated orientation parameters of photo III and the ground coordinates of the tie points:7,8 and 9, as known values. The orientation parameters of photo IV and the ground coordinates of the tie points: 10,11 and 12 are then estimated by the solution of the space resection/intersection of photos III and IV. The estimated values are again updated by the Kalman gain. (g) the second strip starts with the computation of a small block adjustment for photos III, IV of strips one and two. The exterior orientation parameters of photos III and IV in strip one and the ground coordinates of the tie points: 7,8,9,10,11 and 12 are taken as known values. The solution of the equation produces estimated values for the exterior parameters of photos III and IV in strip two and the ground coordinates of the tie points: 14,17,18,21 and 22. (h) the above procedures are repeated until the exterior orientation parameters for photo I in strip two is predicted and updated.

5 (i) since the calculated ground coordinates for the common wing points between the strips one and two are different and the coordinates of these points calculated in strip two is more updated than strip one, and also to find the final improvements for the parameters of the first and second strips, the procedures are traversed in a reverse order from the first photo of strip two to the first photo of strip one as indicated by the direction of the arrows in Fig. 1. It should be noted that in the above formulated solution, the ground coordinates of the points are included in the state vector which means observations are also required for these points. If no observations for these points are available, then two strategies may be adopted: (a) updating the coordinates of the tie points by an indirect way. That is, by space intersection with the updated exterior orientation parameters of each photo pair taken as known values. (b) elimination of the ground coordinates of the points from the state vector. B- GPS observations are available: As stated earlier, with only GPS observations, single strip formation strategy leads to an uncontrolled accumulation of errors in the Y-direction. The reason is the fact that the configuration defect exists in the Y-direction because sequential computation relies only on the orientation elements of the left photo pair for which no observation is available and, thus, error is accumulated for the omega rotation leading to the increase of the residuals for the Y component of the ground coordinates for the tie points. This defect can be suppressed by utilizing a sub-block unit consisting of four, six or more photographs for which a simultaneous adjustment is carried out. The process continues with the sequential connection of the subblocks as is described below (see Fig. 2): (1) bundle block adjustment of photos 1,2,3 and 4 in strips one and two (sub-block I), (2) bundle block adjustment of photos 3,4,5, and 6 in strips one and two (sub-block II) taking the elements of exterior orientation of photos 3 and 4 as known values. (3) the process is continued from the first stage for other sub-blocks. Finally, as stated in section 3.A, a reverse solution from the end to the beginning of the block is also necessary to obtain the last updated values. 4. Experiment description and analysis of results To verify the above stated formulation, a simple preliminary test is conducted on a simulated block consisting of three strips each having ten photographs. The specifications for the simulated data are as follows: Camera frame size = 23 cm 23 cm Focal length = mm Photo scale : 1:25000 Overlap = 60% Sidelap = 10% Ground average height = 200 m omega, phi and kappa are generated by the MATLAB multi-variate normal distribution function with zero mean and the standard deviation equal to ± 2 degrees. Two levels of randomly distributed noise with standard deviations equal to ± 5 m and ± 20 m respectively are also generated and added to the pre-calculated coordinates of the camera projection centers. The corrupted values of the coordinates of the projection centers are then assumed as observed GPS values. The following tests are carried out on these data. Case 1: GPS observations and the six photo sub-block Kalman/adjustment strategy: Bundle adjustment is carried out for sub-blocks consisting of three strips, each strip having two photos. This is accompanied by the sequential block connection and Kalman filter processing as discussed in section 3.B. The vector plot of the residual errors for the ground coordinates of the tie points which are calculated using the known coordinates of the check points, for both data sets ( ± 5 m and ± 20 m noise) are given in Fig.3a and Fig. 3.b respectively. The pattern of the residual vectors (Fig. 3) and the progressive reduction of the magnitude of the averaged residual errors (Figs. 5a and 5b), clearly demonstrates the effectiveness of Kalman filtering process in block adjustment. The block adjustment process is completed by an inverse traverse from the right to the left of the block, as

6 discussed in section 3.A. The vector plot of the residual errors for the completed block adjustment is given in Fig. 6. Case 2: GPS observations and the single strip Kalman/adjustment strategy: Single strip sequential adjustment is performed on each strip separately. As expected and discussed in section 3.B, the error accumulated progressively on the Y component of the ground coordinates of the tie points as indicated in Fig. 4a and Fig. 4b. The mean residual errors for the tie points of the block, indicating an accumulated trend, is presented in Figs. 5c and 5d. 5. Conclusion and future works As demonstrated in the previous section, the proposed sequential method of photo-triangulation has given promising results for the simulated data. This approach in addition to the advantages listed in section 1 is quite flexible in terms of the inclusion of any observations available during the adjustment procedures. These could be GPS and INS data, ground coordinates of the control points measured by a variety of terrestrial methods with different accuracy levels. This is, indeed, a great advantage which is gained by the Kalman filter both in terms of the adaptation of the variety of observations as well as dealing with the low accuracy data. At the moment the following experiments with the sequential Kalman photo-triangulation is being under investigation in our department: - working with the real GPS/INS data acquired during the flight mission. - inclusion of time offset equations (linear and spline, etc.) in the sequential adjustment model. - optimization of the number of photo units in the subblocks for the sequential space resection/intersection. This can be a sub-block consisting of four or more photo units, for which a simultaneous block adjustment is carried out. - implementation of an extended Kalman filter to deal with the complexities encountered with the real data, - adaptation of the proposed model for the newly designed three-line linear array airborne CCD cameras. References: Ackerman, F., Kinematic GPS Control for photogrammetry. Photogrammetric Record, 14(80). American Society of Photogrammetry, The Manual of Photogrammetry, Fourth Edition, Falls Church, VA. Ebner, H Self calibration block adjustment by independent models. Proceedings of annual meeting ASP.

7

8

9

10