Rectification of Aerial 3D Laser Scans via Line-based Registration to Ground Model

Transcription

1 IPSJ Transactions on Computer Vision and Applications Vol (Jul 215) [DOI: /ipsjtcva.7.1] Epress Paper Rectification of Aerial 3D Laser Scans via Line-based Registration to Ground Model Roichi ISHIKAWA1,a) Bo ZHENG1,b) Takeshi OISHI1,c) Katsushi IKEUCHI1,d) Received: March 13, 215, Accepted: April 21, 215 Abstract: The aerial 3D laser scanner is needed for scanning the areas that cannot be observed from the ground. Since the laser scanning takes time, the obtained range data is distorted due to the sensor motion while scanning. This paper presents a rectification method for the distorted range data b aligning each scan line to the 3D data obtained from the ground. To avoid the instabilit and ambiguit of the line-based alignment, the parameters to be optimied are selected alternatel, and the smoothness constraint is introduced b assuming that the sensor motion is smooth. The eperimental results show that the proposed method has the good accurac simulation and actual data. Kewords: Registration, Smoothness constraint, 3D digital archive 1. Introduction Recentl, man large-scale cultural heritage assets have been urgentl desired to be preserved because the deterioration states have become getting worse and worse b natural and man-made breaks. 3D laser scanning is able to pla an important role in the preservation of the cultural heritage assets [1], the large structures over a hundred meters can be digitied in the accurac of a few millimeters with the widel available laser range sensors. The digital 3D models obtained in this wa can be used for the analsis and restoration of the structures. The scanning onl from the ground is not enough due to the large-scale structure of the target objects such as large buildings. It often happens that the surface on the top of the building cannot be observed from an location. Fig. 1 (a) shows an eample of the range data taken from the ground, where the data missing appears on the top of the tower. To obtain the complete 3D model of such a large structure, the aerial laser scanning sstem is required. The aerial laser scanning sstems with inertial sensors or video cameras or both have been previousl proposed. Generall, the aerial scans are distorted due to the sensor movement in the air (See Fig. 1 (b)). The sensors are used to rectif the distortion of the range data b estimating the motion of the sensor. However, there are limitations on the motion estimation accurac b such sensors and cameras. On the other hand, there must be the scans from the ground overlapped with the scans from the aerial scanning sstem. Fig. 1 (b) shows an aerial scan that overlaps with the ground scan shown in Fig. 1 (a). We do not need to take inertial sensors into 1 a) b) c) d) Institute of Industrial Science, The Universit of Toko, Komaba Meguro-ku, Toko , Japan ishikawa@cvl.iis.u-toko.ac.jp heng@cvl.iis.u-toko.ac.jp oishi@cvl.iis.u-toko.ac.jp ki@cvl.iis.u-toko.ac.jp c 215 Information Processing Societ of Japan Fig. 1 (a) (b) (c) (d) Combination of the 3D data from the ground and aerial range data. (a) 3D data from the ground. Missing surfaces eist. (b)distorted aerial range data. (c) Rectified aerial range data. (d) 3D data from the ground and rectified aerial range data. Rectified aerial range data complement the missing surfaces in the data from the ground. account to estimate the motions. Thus, we face the challenge of eliminating the distortion of the aerial scan b using the undistorted scans taken from the ground. Fig. 1 (c) shows the rectified range data of Fig. 1 (b). Fig. 1 (d) shows the merged result of (c) to (a), where the missing areas are filled with the aerial data. In this paper, we propose a method that rectifies the distorted range data based on a line-based alignment algorithm. We assume that range data consists of 3D scan lines obtained sequentiall b the laser scanner. The proposed method aims to estimate the motion of each scan line b aligning it with 3D scans taken from the ground. To avoid the instabilit and ambiguit of the line-based alignment, the motion parameters are estimated based on a nonlinear alternative optimiation. The smoothness constraint is also 1

2 IPSJ Transactions on Computer Vision and Applications Vol (Jul 215) Input Initial alignment Line based registration Result Distorted 3D range data 3D data from the ground Fig. 2 Method overview. Inputs are aerial data and ground data. First, the initial position of aerial data is given b rigid alignment. Second, aerial data is processed to line based registration method and finall we obtain rectified result. introduced b the assumption that the sensor motion is smooth because the sensor is mounted on the balloon. The eperimental results show that the proposed method has good accurac. 2. Related work Our work is related to four research streams in the literature. Structure from Motion (SfM). To recover the sensor motion, SfM-based methods using 2D cameras are popular to use, since the relative pose and position of each frame can be calculated out b using 2D image features [2]. For eample, a famous and robust implementation is proposed in [4]. However, in the case of digital archive, the absolute scale of the range data, which is not eas to get for SfM, is required. SLAM++. SLAM++ [7] is proposed to robustl estimate the sensor motions using depth sensor, suppose 3D priors are observed online and used to improve the localiation. This method is ver useful for robotics or AR applications. However, our method focuses on the high-accurac 3D line reconstruction. It faces more challenging problem as how to match a 3D line to a 3D prior. Non-rigid registration. Man methods (e.g., [3], [6], [1], [11], [12]) have been proposed in the field of non-rigid registration, which geometricall solve the matching problem between the deformed models. However, in this paper, we take the temporal information into account of the deformation (rectified from sensor motion) which is more reasonable in laser scanner case. Another notable work is proposed in [9] which registers 2D curves to a 3D shape encoded b an implicit polnomial and shows efficienc for medical image processing. Aerial Laser Scanning. The method most related to our work is the method proposed b Banno et al., [5], which combines video cameras and range sensor into the scanning sstem and SfM techniques are used in the motion estimation algorithm. In this paper, we onl consider the information obtained from range sensor. Thus our method is independent to an other third-part sensors for saving hardware cost, and also it avoids the calibration problems which often happened in sensor fusion sstems. 3. Method 3.1 Overview The proposed method is based on three assumptions: 1) distortion of one scan line can be ignored because one line scanning time is short (approimate.5 sec); 2) 3D data measured from on the ground is given in advance. This assumption is reasonable because the purpose of this research is to complement the part of data that cannot be measured from the ground; 3) sensor motion is smooth. On these assumptions, position parameters are set to each scan lines and registration is processed. To avoid the instabilit and ambiguit of the line-based alignment, smoothness constraint is imposed on between scan lines. Fig. 2 shows the overview of our method. Aerial scan range data and 3D data from the ground are inputs. First, the initial position of aerial data is given b rigid alignment [8]. Second, aerial data is processed to line based registration method and finall we obtain rectified result. 3.2 Line based registration Initial alignment Distorted range data is aligned to 3D data from the ground for obtaining initial position. This phase is rigid registration. We used fast alignment method presented in [8] in this phase. This method is fast because of searching closest point along to the view direction Motion model of scan line Fig. 3 shows a motion model of scan line. Each line is located on the local coordinates of the range sensor at first. The Airial range data i-1 i i+1 laser scanner 3D data from the ground Fig. 3 c 215 Information Processing Societ of Japan Mosion model of scan line 2

3 IPSJ Transactions on Computer Vision and Applications Vol (Jul 215) Without smoothness constraint Estimated motion of the sensor. sensor motion is not taken into consideration about raw data, therefore each line is deviating each other as moving amount of sensor. Each line is given 6 parameters indicating sensor position, translation parameters T = (,, ) and rotation parameters r = (,, ) (,, ais rotation). The i-th line is firstl transformed b rotation matri Ri and translation vector ti calculated from 6 parameters of i-th line, (i, i, i ) and (i, i, i ), and second, is transformed b R, t obtained b initial alignment. 3.3 Optimiation Cost function Our cost function consists of 2 terms: 1) the distance error term and 2) the smoothness constraint term. The distance error term is based on ICP algorithm. The distance error of i-th line Edi is as follow X sj Edi = a2 ρ( 2 ), (1) a j where ρ() = log(1 + ) is the loss function and a is a scale value, (2) Here, pij is j-th point of i-th line, and pij is closest point of transformed point of pij b R, t, Ri, ti in the data from the ground. The smoothness constraint term E smooth is defined as follow, Z d2 T d2 r E smooth = (λ λ2 2 2 )dt, (3) dt dt This equation is based on the assumption that sensor s acceleration fluctuates smoothl. c 215 Information Processing Societ of Japan Fig. 5 s j = pij (R(Ri pij + ti ) + t) Our method Our method Without smoothness constraint 6. 6 Estimated motion of the sensor. First row: Error visualiation images. Second row: Histogram of error.our method 1: Optimiing 6 parameters. Our method 2: Optimiing selected 3 parameters alternatel 8. 3 Fig Our method Our method 1 Initial data Consequentl, Global cost function is 1 X E= ( Edi ) + we smooth, N i (4) where N is the number of point, w is a coefficient determined eperimentall Decremental smoothness constraint Smoothness constraint is introduced for reducing instabilit and ambiguit. However, Eq. 3 indicates that the ideal is that variation of sensor s acceleration is. Practical variation of sensor s acceleration is not and a parado eists between practical sensor movement and Eq. 3. Therefore coefficient w in Eq. 4 is graduall decreased Parameters selection Variables to be optimied are 6 parameters of each line position. However, in some phases, onl 3 parameters are optimied simultaneousl and parameters to be optimied are selected alternativel because the lower the stabilit and the ambiguit of the line-based alignment. (,, ) are initiall selected as parameters to be optimied because these parameters are tend to var wider than other 3 parameters. This tendenc proceeds from balloon movement. Therefore it is reasonable that 3 number o f lines parameters are optimied globall and simulteniousl at once to minimie the cost function, and parameters to be optimied are selected alternatel. This scheme is especiall effective in case that ambiguit of scan line is large. As a non-linear least square solution of this optimiation problem, Levenberg-Marquardt Method is selected. 3

4 IPSJ Transactions on Computer Vision and Applications Vol (Jul 215) Ground data Distorted data Rectified data Error visualiation Fig. 6 Other results of rectification. From left to right, ground data, distorted data, rectified data, error visualiation. 4. Eperimental Results In this eperiment, we process 2 methods; 1) Optimiing 6 parameters simultaneousl and 2) Optimiing selected 3 parameters alternatel. In each method, the following steps are processed and each step finishes when cost function is convergence or iteration times reached the specified value. Optimiing 6 parameters of each lines simultaneousl: 1. w is w is w is no smoothness constraint. Optimiing selected 3 parameters alternatel: 1. (,, ) are optimied, w is (,, ) are optimied, w is (,, ) are optimied, w is (,, ) are optimied, w is (,, ) are optimied, w is (,, ) are optimied, no smoothness constraint. 4.1 Simulation data In this eperiment, we use two range data from the ground: one is used as a reference data and another is used as a distorted data. The distorted data is made b transforming each scan line with preliminar determined parameters. These parameters are manuall set to imitate balloon motion. Therefore we can evaluate rectified data b comparing the estimated parameters and the preliminar determined parameters as true values. Results are showed in Fig. 7 and Fig. 8. Graphs in the first column show true values of line positions and graphs in the second and third columns show error of the estimated motion defined as (T rue value) (Estimated value). Therefore these graph indicate that the closer to error is, the more acculate estimated parameter is. From Fig. 8, it is possible to sa that our methods succeed in estimating true positions for the most part. Comparing our method 1 and our method 2, there is no large tendenc that where lines c 215 Information Processing Societ of Japan which have large error is located. The results of method 1 are seemingl more accurate, however processing time of method 2 tend to be shorter because the number of parameter to be optimied simultaneousl is half as large as the method 2, therefore optimiation of selected 3 parameters alternatel has the room for practical use. 4.2 Actual aerial data As evaluation methods in this section, error visualied 3D images and histogram of errors are selected. The error is defined b Euclidean distance between a point in aerial data and its closest point in 3D data from the ground. Fig. 4 shows comparison results. In the error visualiation image, each point is colored depending on its error (See a color bar at the top left of error visualiation image). Blue point has small error and white point is large error point because of wrong estimation or missing ground data. Our method achieved the aerial data rectification with less error than without smoothness constraint. Bottom row of Fig. 5 shows the position of line. Each line is scanned in time order, therefore this graphs indicate estimated crude sensor motions. From Fig. 5, in case of imposing smoothness constraint, estimated result is more smooth than without constraint. Comparing to results b method 1 (optimiing 6 position parameters simultaneousl) and method 2 (selected 3 parameters alternativel), we will see that the method 2 succeed to estimate position of lines which method 1 cannot rectified. (In red circle in Fig. 5). However, the result of method 1 has more points whose error is small than the result of method 2 (See the histograms in Fig. resultcompare). Therefore, it ma be possible to achieve more acculate result b finding appropriate combination of steps. Fig. 6 shows results of other data. The results in first 3 rows 4

5 IPSJ Transactions on Computer Vision and Applications Vol (Jul 215) (a) Fig. 7 (b) 3D image of simulation data. (a) Reference data, (b)distorted data, (c) Our method 1, (d) Our method 2 (c) (d) True value Our method 1 Our method Fig. 8 Estimated position of the line. are successful cases. The result in the bottom row is comparable failure case. In this case, ais rotation movement of sensor is comparabl drastic and some points in aerial range data, in particular far points from sensor, are located on far coordinates from correct place initiall. 5. Conclusion With smoothness constraint considering time domain, our linebased alignment method was dramaticall improved and achieved well-rectified results nevertheless onl geometrical information is used. Accurac of rectified results is depends on 2 elements: one is shape characteristic and another is a magnitude of the distortion. Shape characteristic is important information to reduce ambiguit and to determine position of line. Therefore uneven object, like Angkor-Wat, is compatible with our method compared to flat object. A magnitude of the distortion is intensit of sensor motion during measurement and it depends on the intensit of the wind. Therefore it is of course ideal to do measurement under weak wind conditions. However, a lot of measurements are needed to archive a large cultural structure, hence it is difficult to alwas obtain data measured under the ideal conditions from a cost and schedule perspective. Thus it is needed to rectif even range data with large distortion measured under the strong wind conditions. Our method shows good performance to an aerial data measured under the calm wind conditions. However when distortion of aerial data is drastic, our method cannot shows good performance. For obtaining more accurate results,we are thinking of implementing another scheme involving sensor motion, or use other data, such as reflectance information. Acknowledgments This work is supported in part b Netgeneration Energies for Tohoku Recover (NET), MEXT, Japan. References [1] K. Ikeuchi and K. Hasegawa and A. Nakaawa and J. Takamatsu and T. Oishi and T. Masuda, Baon digital archival project, Proc. of the Tenth International Conference on Virtual Sstems and Multimedia, pp , 24. [2] T. Carlo, K.Takeo, Shape and motion from image streams under orthograph: a factoriation method, International Journal of Computer Vision, pp , [3] K. Fujiwara, K. Nishino, J. Takamatsu, B. Zheng and K. Ikeuchi, Locall Rigid Globall Non-rigid Surface Registration, Proc. of IEEE International Conference on Computer Vision (ICCV), pp , 211. [4] Furukawa, Y.; Ponce, J., Accurate, Dense, and Robust Multiview Stereopsis, Pattern Analsis and Machine Intelligence, IEEE Transactions on, vol.32, no.8, pp , 21. [5] A. Banno, T. Masuda, T. Oishi, and K. Ikeuchi. 28. Fling Laser Range Sensor for Large-Scale Site-Modeling and Its Applications in Baon Digital Archival Project, International Journal of Computer Vision, pp , 28. [6] T. Masuda, Y. Hirota, K. Nishino and K. Ikeuchi. Simultaneous determination of registration and deformation parameters among 3D range images. Proc. of 5th International Conference on 3-D Digital Imaging and Modeling (3DIM25), pp , 25. [7] Renato F. Salas-Moreno, Richard A. Newcombe, Hauke Strasdat, Paul H. J. Kell and Andrew J. Davison, SLAM++: Simultaneous Localisation and Mapping at the Level of Objects, Proc. of Computer Vision and Pattern Recognition (CVPR) [8] T. Oishi, A. Nakaawa, R. Kuraume and K. Ikeuchi, Fast Simultaneous Alignment of Multiple Range Images using Inde Images, Proc. of 5th International Conference on 3-D Digital Imaging and Modeling (3DIM25), pp , 25. [9] B. Zheng, R. Ishikawa, T. Oishi, J. Takamatsu, K. Ikeuchi, A Fast Registration Method Using IP and Its Application to Ultrasound Image Registration, IPSJ Transactions on Computer Vision and Applications, vol.1, pp , 29. [1] B. Brown, S. Rusinkiewic, Gloabal Non-Rigid Alignment of 3-D Scans. ACM Transaction on Graphics (Proc SIGGRAPH), vol.26, No.3, 27. [11] R. Sagawa, K. Akasaka, Y. Yagi, H. Hamer, Elastic Convolved ICP for the registration of deformable objects, IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp , 29. [12] Q. Zhou, S. Miller, V. Koltun, Elastic Fragments for Dense Scene Reconstruction, IEEE International Conference on Computer Vision (ICCV), pp , 213. c 215 Information Processing Societ of Japan 5