Development of a Test Field for the Calibration and Evaluation of Kinematic Multi Sensor Systems DGK-Doktorandenseminar Graz, Austria, 26 th April 2017 Erik Heinz Institute of Geodesy and Geoinformation University of Bonn, Germany
Outline I. Motivation II. Calibration of mobile laser scanning systems a. Plane-based calibration approach b. Simulation and analysis of different plane setups c. Application to a mobile laser scanning system III. Outlook and Conclusions a. Evaluation of trajectory estimation b. Evaluation of kinematic object acquisition Erik Heinz 18. Ingenieurvermessungskurs 2017, Graz, Austria Page 2
Motivation Paradigm change in engineering geodesy The 3D surveying of the environment has changed manual vs. automatic single points vs. area-based static vs. kinematic single sensor vs. multi sensors [1] [2] Kinematic multi sensor systems [3] available operative established Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 3
Motivation Challenges of kinematic multi sensor systems Georeferencing (GNSS, IMU, odometer etc.) data fusion for trajectory estimation in a filter algorithm integration of stochastic models Object acquisition (laser scanners, cameras etc.) considering object characteristics, configuration, image processing, integration of stochastic models Time synchronization time stamping of the observations within a common time reference Calibration intrinsic calibration: calibration of individual sensor deviations extrinsic calibration: calibration of lever arms and boresight angles 3D point clouds Σ pp?? How good are the 3D point clouds in terms of precision and accuracy? result of complex processing steps Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 4
Motivation What do we need for stochastic modeling of Σ pp? functional model f complex and partially unknown observations l and calibration parameters c too many Σ pp distribution functions Σ ll and Σ cc mixture and in general not Gaussian Full covariance matrix Σ pp not available, we need: empirical determination of Σ pp (not trivial!!!) evaluation procedures and criteria calibration procedures and models [4] Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 5
Goals of this thesis Key issue of this thesis What are suitable methods, models and criteria for a good calibration and a comprehensive evaluation of kinematic multi sensor systems and how has a test field to be designed to allow for this? [4] Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 6
Goals of this thesis [5] Development of a test field Calibration objects (planes, cylinders, cones) Reference trajectory (railway, rail vehicle, ) Reference point field (fixed points, pillars, ) Evaluation objects (planes, cylinders, cones) Environment (buildings, digital model) Test field Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 7
Goals of this thesis [5] Calibration objects (planes, cylinders, cones) Reference trajectory (railway, rail vehicle, ) Reference point field (fixed points, pillars, ) System calibration Evaluation objects (planes, cylinders, cones) Evaluation trajectory Evaluation kinematic object acquisition Environment (buildings, digital model) Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 8
Outline I. Motivation II. Calibration of mobile laser scanning systems a. Plane-based calibration approach b. Simulation and analysis of different plane setups c. Application to a mobile laser scanning system III. Outlook and Conclusions a. Evaluation of trajectory estimation b. Evaluation of kinematic object acquisition Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 9
Calibration parameters Calibration of mobile laser scanning systems Calibration of single sensors GNSS IMU 2D laser scanner Installation position and orientation of sensors lever arms boresight angles IMU GNSS Scanner intrinsic calibration IMU GNSS Scanner extrinsic calibration [6] [7] [8] Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 10
Mobile Laser Scanning System GNSS/IMU unit imar inav-fji-lsurv multi-frequency GNSS, fiber-optic gyroscopes and accelerometers σ Position = cm, σ Attitude < 0.025 GNSS Master Station Mobile Laser Scanning System 2D laser scanner 2D laser scanner Z+F Profiler 9012 A phase-based 2D laser scanner point accuracies of mm GNSS/IMU unit GNSS Antenna Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 11
Calibration Approach Problem: Determination of calibration parameters lever arm Δx, Δy, Δz and boresight angles α, β, γ between GNSS/IMU unit and 2D laser scanner range offset d 0 of the 2D laser scanner x e y e z e = t x t y t z n + R b φ, θ, ψ x y z b + R s α, β, γ 0 (d + d 0 ) sin b (d + d 0 ) cos b georeferenced scan point position/attitude (GNSS/IMU unit) extrinsic calibration scan point (laser scanner) + intrinsic calibration Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 12
Calibration Approach Approach: Calibration field with reference planes Setup of reference planes 1) georeferencing (e.g. TLS + GNSS control points) 2) static scanning with the MSS 1 2 Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 13
Calibration Approach Solution: Parameter estimation georeferenced TLS points must fulfill the plane equations g 1 : s e sin z e cos h e s e sin z e sin h e s e cos z e T nx n y n z d P = 0 observations parameters georeferenced MSS points must fulfill the plane equations g 2 : t x t y n + R b φ, θ, ψ t z x y z b + R s α, β, γ 0 (d + d 0 ) sin b (d + d 0 ) cos b T nx n y n z d P = 0 adjustment of g 1 and g 2 in a Gauß-Helmert model Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 14
Outline I. Motivation II. Calibration of mobile laser scanning systems a. Plane-based calibration approach b. Simulation and analysis of different plane setups c. Application to a mobile laser scanning system III. Outlook and Conclusions a. Evaluation of trajectory estimation b. Evaluation of kinematic object acquisition Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 15
Calibration Results Simulation Simulation of different configurations What is a suitable configuration (setup of reference planes and how they are scanned with the mobile system) to obtain good calibration parameters (small variance, uncorrelated)? Simulation and analysis of 4 configurations, each with: 8 reference planes 6 stations of the mobile laser scanning system Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 16
Calibration Results Simulation Simulation of configurations F 1a and F 1b Configuration F 1a : identical attitude of MSS, plane rotation of 45 45 45 45 45 15-20 m Configuration F 1b : diametrical attitude of MSS, plane rotation of 45 45 45 45 45 15-20 m Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 17
Calibration Results Simulation Simulation of configurations F 2a and F 2b Configuration F 2a : identical attitude of MSS, plane rotation of 30 30 30 30 30 15-20 m Configuration F 2b : diametrical attitude of MSS, plane rotation of 30 30 30 30 30 15-20 m Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 18
Calibration Results Simulation Covariance matrix Σ ll of the observations manufacturer information no correlations TLS + GCP s h z 1σ (STD) 0.001 m 0.0025 0.0025 GNSS/IMU t x, t y, t z Φ, θ, ψ 1σ (STD) 0.01 m 0.01 2D scanner d b 1σ (STD) 0.0003 m 0.02 Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 19
Calibration Results Simulation Covariance matrix Σ xx of the parameters parameter accuracies depend on the accuracy of the GNSS/IMU unit and the number of MSS stations σ β and σ d0 vary for different configurations σ x, y, z = 0.01 m 6 = 0.0041m σ α,β,γ = 0.01 6 = 0.0041 σ Δx σ Δy σ Δz σ α σ β σ γ σ d0 F 1a ( I ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0046 0.0041 0.02 mm F 2a ( I ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0070 0.0041 0.02 mm F 1b ( I + II ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0042 0.0041 0.01 mm F 2b ( I + II ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0058 0.0041 0.01 mm Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 20
Calibration Results Simulation Correlation matrix of the parameters parameter correlations: diametrical scannning decorrelation of β and d 0 improving of σ β and σ d0 F 1a : F 1b : F 2a : F 2b : not scanned diametrically scanned diametrically Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 21
Calibration Results Simulation Why is σ β generally better in F 1 than in F 2? σ Δx σ Δy σ Δz σ α σ β σ γ σ d0 F 1a ( I ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0046 0.0041 0.02 mm F 2a ( I ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0070 0.0041 0.02 mm F 1b ( I + II ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0042 0.0041 0.01 mm F 2b ( I + II ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0058 0.0041 0.01 mm Question: How sensitive are the reference planes with respect to changes in the calibration parameters? Strategy: Simulation of parameter deviations for the boresight angles in F 1 and F 2 : α = 0.03 / β = 0.03 / γ = 0.03 Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 22
Calibration Results Simulation Sensitivity of the reference planes F 1 and F 2 Sensitivity in Field F 1 Sensitivity in Field F 2 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 23
Calibration Results Simulation Findings of the simulations configuration in the calibration field is crucial for the quality of the calibration results, which can worsen due to parameter correlations sensitivity of the reference planes analysis allows for a refinement of the network configuration Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 24 [9]
Outline I. Motivation II. Calibration of mobile laser scanning systems a. Plane-based calibration approach b. Simulation and analysis of different plane setups c. Application to a mobile laser scanning system III. Outlook and Conclusions a. Evaluation of trajectory estimation b. Evaluation of kinematic object acquisition Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 25
Calibration Results Real System Realization of configuration F 1b TLS scan of reference planes (size of 0.6 x 0.9 m) georeferencing using 5 GCPs (statically measured for 2.5 h, residuals 2 mm) MSS scans of the planes (30 diametrical stations) static scanning (about 60 s per station: reduce noise, eliminate timing errors) GCP Reference Planes 3D Laser Scanner Mobile Laser Scanning System (Diametrical Stations) Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 26
Calibration Results Real System Calibration results confirm the simulation results Σ ll due to simulations (σ Pos = 0.01 m, σ Att = 0.01 ) accuracy of the calibration parameters: σ x, y, z = 0.01 m 30 = 0.0019m σ α,β,γ = 0.01 30 = 0.0019 no parameter correlations Δx Δy Δz α β γ d 0 Value -0.7957 m 0.0448 m 0.1731 m 0.2170 0.1799-0.1817 0.70 mm STD (1σ) 0.0019 m 0.0019 m 0.0019 m 0.0019 0.0019 0.0019 0.01 mm Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 27
Calibration Results Real System Repeatability of the calibration results subsampling of the observations into 5 blocks each block contains 6 stations from the original 30 stations Stations Δx Δy Δz α β γ d 0 all -0.7957 m 0.0448 m 0.1731 m 0.2170 0.1799-0.1817 0.70 mm 1-6 -0.7960 m 0.0451 m 0.1736 m 0.2143 0.1894 (-0.2036 ) 0.60 mm 7-12 -0.7961 m 0.0451 m 0.1730 m 0.2190 0.1636-0.1750 0.74 mm 13-18 -0.7951 m 0.0447 m 0.1732 m 0.2170 0.1782-0.1780 0.57 mm 19-24 -0.7953 m 0.0444 m 0.1713 m 0.2189 0.1761-0.1754 0.76 mm 25-30 -0.7961 m 0.0448 m 0.1748 m 0.2152 0.1885-0.1743 0.65 mm Max-Min 0.0010 m 0.0007 m 0.0035 m 0.0047 0.0258 0.0037 0.19 mm Simulation STD (1σ) 0.0041 m 0.0041 m 0.0041 m 0.0041 0.0042 0.0041 0.01 mm Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 28
Permanent calibration field Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 29
System Evaluation Comparison of two kinematic scans two kinematic scans of the Poppelsdorfer Schloss opposite direction of movement mean distance of 15 20 m Error propagation of: intensity-coded observations: Σ ll calibration: Σ xx Expected accuracy of the point cloud: σ 3D = 2 cm. 25 m 20 m Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 30
System Evaluation Comparison of two kinematic scans two kinematic scans of the Poppelsdorfer Schloss M3C2-comparison in CloudCompare deviations δ have an RMS of 2.75 cm ( 2 cm 2) systematic effects at the facade (due to trajectory) # 3000 2000 1000 25 m 0 δ [cm] -7.5-5 -2.5 0 2.5 5 7.5 Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 31
Outline I. Motivation II. Calibration of mobile laser scanning systems a. Plane-based calibration approach b. Simulation and analysis of different plane setups c. Application to a mobile laser scanning system III. Outlook and Conclusions a. Evaluation of trajectory estimation b. Evaluation of kinematic object acquisition Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 32
Evaluation trajectory Rail-bound reference trajectory variation in all 6 DoF rail vehicle for adaption of systems repeated driving of the trajectory tracking of multi sensor systems equipment (total station, prism, offsets,..) stability of the reference trajectory time synchronization [4] [10] Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 33
Evaluation object acquisition Reference objects determination of position, orientation and geometry of reference objects total station, laser tracker, TLS, reference point field kinematic acquisition (e.g. from the reference trajectory) comparison: point cloud vs. reference Question: Where have the reference objets to be placed? Idea: Simulations like for the plane-based calibration [4] Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 34
Evaluation object acquisition Reference objects Example: Simulation of deviations in the boresight angles while scanning a setup of reference planes from a distance of ~ 5 m β = 0.1 γ = 0.1 8 8 4 0 motion 0 motion -8 mm mm Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 35
Conclusions [5] Reference trajectory (railway, rail vehicle, ) Calibration objects (planes, cylinders, cones) Finished / In Progress In Progress To Do Reference point field (fixed points, pillars, ) System calibration Evaluation objects (planes, cylinders, cones) Evaluation trajectory Evaluation kinematic object acquisition Environment (buildings, digital model) Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 36
Peer-Reviewed Relevant Publications Heinz, E., Eling, C., Wieland, M., Klingbeil, Kuhlmann, H. (2017) Analysis of Different Reference Plane Setups for the Calibration of a Mobile Laser Scanning System, In: Lienhart, W. (Hrsg.): Ingenieurvermessung 17, Beiträge zum 18. Internationalen Ingenieurvermessungskurs, Graz, Österreich, S. 131-145, Wichmann Verlag, Berlin, Offenbach. Heinz, E., Eling, C., Wieland, M., Klingbeil, Kuhlmann, H. (2015) Development, Calibration and Evaluation of a Portable and Direct Georeferenced Laser Scanning System For Kinematic 3D Mapping, In: Journal of Applied Geodesy 9 (4), pp. 227-243. Non-Peer-Reviewed Heinz, E., Eling, C., Wieland, M., Klingbeil, Kuhlmann, H. (2016) Development of a Portable Mobile Laser Scanning System with Special Focus on the System Calibration and Evaluation, 5th International Conference on Machine Control & Guidance (MCG), Vichy, France, October 5-6, 2016. Eling, C.; Klingbeil L.; Wieland, M.; Heinz, E.; Kuhlmann, H. (2015) UAV Real-time - Data Use in a Lightweight Direct Georeferencing System. GPS World, November Issue, pp. 44-49. Klingbeil, L., Eling, C., Heinz, E., Wieland, M., Kuhlmann, H. (2015) Direct Georeferencing for Portable Mapping Systems In the Air and on the Ground, 9th International Symposium on Mobile Mapping Technology (MMT 2015), Sydney, Australia, December 9-11, 2015. Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 37
Thank you for your attention Questions or comments? M.Sc. Erik Heinz Institute of Geodesy and Geoinformation University of Bonn, Germany Tel.: +49 228 / 73-3034 Email: e.heinz@igg.uni-bonn.de Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 38
List of Figures [1] http://www.leica-geosystems.de/thumbs/originals/mihq_2011.jpg (accessed: 24/04/17) [2] https://i.ytimg.com/vi/zlpojwhxno8/hqdefault.jpg (accessed: 24/04/17) [3] http://www.positionpartners.com.au/product_images/260_1442544472.jpg (accessed: 24/04/17) [4] https://cdnsc.wallsheaven.com/photos/c30226497_220.jpg (accessed: 24/04/2017) [5] GoogleEarth (accessed: 24/04/2017) [6] http://www.geotron-moscow.ru/images/gps.png (accessed: 24/04/17) [7] http://www.imar-navigation.de/media/zoo/images/imu-fsas-ng_2_b988afc5632521b628c8 af9462365ee5.jpg (accessed: 24/04/17) [8] http://www.zf-laser.com/typo3temp/pics/profiler_9012_2d_laser_scanner_44c6933d71.png (accessed: 24/04/17) [9] https://t4.ftcdn.net/jpg/00/89/75/09/240_f_89750910_ybv8d9yfdmbqmyp5padcc8jcy dyzzszn.jpg (accessed: 24/04/17) [10] https://cdn.shopify.com/s/files/1/0542/7477/products/prism-leica-grz122-withinstrument.jpg?v=1416481986 (accessed: 24/04/2017) Erik Heinz DGK-Doktorandenseminar 2017, Graz, Austria Page 39