An Automatic Laser Scanning System for Accurate 3D Reconstruction of Indoor Scenes

Transcription

1 An Automati Laser Sanning System for Aurate 3D Reonstrution of Indoor Senes Danrong Li, Honglong Zhang, and Zhan Song Shenzhen Institutes of Advaned Tehnology Chinese Aademy of Sienes Shenzhen, Guangdong Provine, China {dr.li; hl.zhang1; Duhu Man and Mark W. Jones Computer Siene Department Sansea University, UK Singleton Park, Sansea, SA2 8PP, Wales, UK {m.duhu; Abstrat 3D reonstrution of indoor senes is important for lots of appliations like SLAM (Simultaneous Loalization and Mapping), VR (virtual reality) and AR (augmented reality) et. This paper presents a preise and lo-ost 3D reonstrution system, hih onsists of a laser emitter, monoular ameras and a turntable. The system is firstly alibrated to determine the intrinsi and extrinsi parameters of the amera and laser plane. By rotating the sanning devie, a serial of laser strip images an be obtained. With the proposed point loud registration algorithm, multiple 3D profiles of the target sene an be omputed preisely and aligned automatially. As a result, a omplete 3D model of the indoor sene ith olor information and high auray an be obtained. In the experiments, some real senes are reonstruted and evaluated from the aspets of both visual quality and measurement auray. Index Terms Strutured light system, 3D reonstrution, alibration, point loud registration. I. INTRODUCTION Sene modelling is an important aspet of game design, movie making and many VR and AR appliations. The basi modelling approah is to use some programming languages suh as X3D, VRML, 3DS MAX and Maya et. [1]. Artifiial modeling is time-onsuming and laborious, and modeling auray and realism annot be guaranteed, espeially for some irregular objets in real sene onstrution. On the ontrary, 3D sanning tehnologies an aurately apture 3D senes in a short time and thus are idely used for 3D modelling appliations. There are many kinds of 3D reonstrution equipment and methods, hih an be divided into to types: ontat type and non-ontat type. The typial measuring instrument of ontat type is the three oordinate measuring instrument [2], hih has high preision, but is not suitable for the modeling of the sene or the preision of the objet. Non-ontat type reonstrution inludes 3D reonstrution, photometri stereo [3, 4], stereo vision method [5, 6], SFS (Struture from Motion) [7], strutured light [8, 9] and so on. These have been ell developed in reent years. This system uses a lo ost three-dimensional reonstrution of strutured light, hih ompared to some of the strutural light reonstrution equipment, is muh more ost effetive. Compared ith binoular vision and the SFM algorithm, the feature points are easier to detet, and it is not neessary to measure 3D data by laser radar. Some reent RGBD ameras also have been used for the 3D reonstrution of indoor senes [1-12], like Kinet, TOF and RealSense devies et. But, the reonstrution auray is loer than approahes hih are based on visual triangulation, partiularly point loud registration. The diagram of the proposed automati laser sanning system is shon in Fig. 1. The system onsists of four omponents: one amera, one turntable, one laser-line projetor, and mounting brakets. The laser and amera are fixed and mounted on the rotation table. During sanning, the laser-line projetor remains open, the turntable starts from, and the image ith the laser-line is aptured every α degrees. The program alulates the point loud on the interseting line of the laserline plane and the surfae of objets in the urrent amera oordinate frame aording to the image aptured by the amera. The turntable rotates 36/α times to omplete a omplete san. With the proposed point loud registration algorithm, point louds omputed at various sanning positions an be automatially integrated by the transformation matrix that is alulated from system alibration. With a full rotation, a 3D model of the room ith olor texture an be obtained ithout user interations. Mounting Brakets Laser-line Projetor Turntable Laser Plane Camera Objet This ork as supported in part by the National Natural Siene Foundation of China (Nos , U ), Shenzhen Siene Plan (JCJ , JSGG ) and The CAS Key Laboratory of Human-Mahine Intelligene-Synergy Systems. Duhu Man is supported by a Sêr Cymru COFUND felloship. Stripe Image Fig. 1. Illustration of the proposed automati 3D laser sanning system.

2 II. MODELING AND CALIBRATION OF THE 3D SSTEM The reonstrution model presented in this paper is based on the perspetive projetion and pinhole models. The relative oordinate frames are defined as follos. A point in the orld oordinate frame an be transformed from the orld oordinate frame to the amera frame, and then to the image frame. Ideally, there is no distortion in the lens, and the pinhole amera model is shon in Fig. 2. There are four oordinate systems defined in the 3D sanning system, i.e., O n-x n n: the 2D normalized image frame, O n-x u u: the 2D image oordinate frame, O -X Z : the 3D amera oordinates frame, and O - X Z : the 3D orld oordinate frame. O is the projetion enter of the amera and the Z -axis is the optial axis of the amera lens. X Z O [ R T ] O u Ou X u X Pn On P (u, v ) n Objet Fig. 2. Geometrial model of the 3D laser sanning system [13]. The transformation from the amera oordinate frame to the image frame an be expressed as: u 1/ s u f x 1/ y Z Z v s v f X X a u b v A Z Z X n Z X here s x and s y represent the horizontal and vertial pixel pith on the sensor, and (u, v ) is the priniple point. The image plane is loated at a distane of f behind the projetion enter. A is alled the intrinsi parameters matrix, α and β are the effetive foal length in pixels of the amera in the x and y diretions, and desribes the skeness of the to axes of the image. The transformation from an image point (u, v) to the orld oordinate 3D point (X,, Z ) an be simplified as: (1) X X u v R t A I A Z I H (2) T Z here is a 4 4 transformation matrix hih relates the H orld oordinate frame to the amera oordinate frame, also alled the extrinsi parameters matrix. t is a 3 1 translation vetor and R is a 3 3 orthogonal rotation matrix. R and t determine the position of the amera ith respet to the orld, I is the 3 3 identity matrix, and is a 3 1 zero matrix. When onsidering lens distortion of amera, there are deviations beteen the atual projetion points (u, v ) and the theoretial projetion points (u, v) in the image [14]. For most lenses, the distortion model an be approximated as: u' 2 u = v' k(u +v ) v here k models the magnitude of the radial distortions. If the distortion is pinushion-shaped, k is positive, hile for barrelshaped it is negative. To desribe the mapping of a 3D point from the orld oordinate frame to the 2D image oordinate frame, the intrinsi and extrinsi parameters should be alibrated. Photogrammetri alibration and self-alibration are the to major ategories in the field of alibration. The photogrammetri method needs to use a preision alibration blok [15], and the intrinsi and extrinsi parameter matries of the amera are alulated by establishing the orrespondene beteen the points on the alibration blok and the image points. Self-alibration [16] estimates intrinsi and extrinsi parameter matries based on the orrespondene beteen image points and does not require a alibration objet ith knon 3D geometry. Zhang [17] proposed a flexible and high preision approah for amera alibration, requiring only the amera to observe a planar pattern hih an easily be printed and plaed on a rigid planar surfae shon at a fe (at least to) different orientations. Thereby radial lens distortion is modeled. Using his proposed proedure, the intrinsi and extrinsi parameter matries of alibration an be solved, and the distortion orretion an be alulated analytially. It is evident from (2) that only knoing the image projetion point (u, v) in the image oordinate frame annot solve the 3D point (X,, Z ) in the 3D amera oordinate frame, beause any point on the laser-line projetion appearing ithin the amera field of vie is a point in the laser plane. If the laser plane equation an be solved in the amera frame of referene, the laser plane an be desribed as: (3) ax by z (4) here a, b and are onstants desribing the laser-plane equation. Then the spatial points in the amera frame orresponding to the projetion point (u, v) in the image oordinate frame an be determined by (2) and (4).

3 Many methods exist to alibrate laser light plane. One of the ommon methods is to use a alibration blok, hih is a preisely manufatured objet from hih a fe feature points an be obtained. Abu-Nabah [18] uses the retangular noth alibration blok suessfullyby having its top surfae aligned vertially to the laser plane alloing alibration of the laser line plane equation. To alibrate the strutured light plane, ross-ratio invariane is also a ommon method. Zhou [19] proposed a novel approah to alibrate the laser plane ith free-moving one-dimensional target. The amera oordinates of a ontrol point on the light-stripe an be alulated by the feature-line equation and its distane to the origin is alulated ith its projetive oordinates based on a 1D projetive transformation. The laser plane as then fitted to these points. A alibration approah based on ative vision for a light plane as proposed by Chen [2], here the projetion of parallel lines in the light plane are aptured through translational motions, and the normal vetor of the light plane is alulated from vanishing points. In [21], a simple alibration method for deriving the strutured light plane parameters for elding robots as proposed. The laser irradiates on the alibration plate during amera alibration, beause the points on the intersetion of the laser plane and the surfae of the objet meet the pinhole model. Moreover, aording to Zhang's alibration method, it assumes the model plane is on Z= of the orld oordinate frame, and detets the sub-pixel enter of the laser line ui, v i, i 1,2,3... on the surfae of the alibration plate. Therefore, the orresponding oordinates in the amera oordinate frame an be solved as follo: Z X i r31 tx r32 ty r33t z ( u - u ) r / ( v - v ) r r i 33 i Z ( v v ) (5) i i i (6) Z ( u u ) i i (7) The laser light plane equation (4) an be fitted by using a series of laser lines on the alibration plates ith different orientations as (8). min a, b, d X a Z 1 b X i i 1 Z i By solving (8) via a linear least squares method, the onstants a, b and an be uniquely identified. Therefore, aording to (2) and (4), e an alulate 3D positions of the points lying on the laser strips in the aptured image. III. ALIGNMENT OF POINT CLOUDS AT DIFFERENT SCAN ANGLE (8) In setion II, the reonstruted 3D points are only defined based on the amera referene frame. While the sanning system is rotated, multiple 3D laser lines should be aligned together under the same orld oordinate frame. Therefore, it is neessary to alibrate the turntable. If the installation of turntable and amera is aurate, the rotation vetor oinides ith the rotation axis and the amera oordinate frame ill rotate around the turntable axis as shon by Fig. 3. O rotate axis X Z turntable Á t z X 1 1 Z 1 O 1 O X Fig. 3. The sanning system should rotate around the turntable axis. The amera oordinate frame at a ertain angle α an be onverted to a fixed oordinate frame O-XZ by (9). Z X os sin X1 1 1 Z sin os t z Z Hoever, in pratie, it is impossible to obtain the ideal mounting of braket and amera. The amera oordinate frame doesn t revolve around the vertial support but around a rotation vetor as shon by Fig. 4. It is diffiult to estimate the transition matrix beteen O -X Z and O-XZ by manual measurement. Therefore, e propose a method to obtain the transformation matrix of the amera from different positions to the fixed oordinate system and realize the point loud integration automatially. O turntable rotate axis X Z Á X 1 1 Z 1 O 1 O Fig. 4. The real rotation axis is not oinident ith rotation table axis. X Z H H 1 Z O X R (9)

4 Aording to [17], the alibration plate plane lies on Z= of the orld oordinate frame during alibration. The transformation matrix hih relates the orld oordinate frame to the amera oordinate frame as obtained after alibration. Therefore, if to ameras loated at O and O 1 take a fixed alibration plate respetively, e an get their transformation matries as: X X H (1) Z Z 1 1 X1 X H (11) 1 1 Z 1 Z 1 1 X 1 X X H H Tr (12) Z 1 Z Z After obtaining the amera oordinate frame transformation matrix of a fixed rotation angle, at the angle of a=n α (n=, 1, / a ), the amera oordinate frame an be transformed to the fixed oordinate frame as: Xn X n n Tr (13) Z n Z 1 1 In order to improve the omputation speed, Tr n is stored as a look-up-table. Therefore, the system an take pitures every α degrees. By solving eah point loud, and transferring the point loud to a fixed amera frame, a omplete 3D model an be obtained. IV. EXPERIMENTAL RESULTS The experimental setup onsists of a USB 3. amera ith resolution of pixels, and a lens ith a 6mm foal length. A red line laser ith an emitting angle of about 9 is used. After the sanning system is installed, the amera an be alibrated using the method of Zhang. The alibration took a total of 23 alibration plate images, of hih the last four are used to fit the laser plane equation. The last to are used to estimate the rigid transformation matrix of amera before and after rotation. In order to get a more dense point loud, this experiment takes α=.1. The intrinsi parameters matrix A and the transformation matrix Tr obtained by the alibration of amera are shon in Table I. TABLE I INTRINSIC PARAMETERS MATRIX Parameters α β u v Value e e e- 6 Tr The alibration matrix of eah alibration plate is obtained. The enter points of laser line are extrated from the last four graphs respetively as shon in Fig. 5. Aording to the equations (5) - (7), e an get the 3D points (X i, i, Z i), i=1, 2...n in the amera oordinate system. Then, aording to these points, equation (8) is used to fit the laser plane to obtain the oeffiients of the laser plane equation. The laser plane oeffiients are shon in Table II. Laser plane oeffiient Fig. 5. Laser stripe extration TABLE II LASER PLANE COEFFICIENT Value a b With the laser plane equation, simultaneously the amera model an be adopted to solve the 3D oordinates of the laser line. The point of the inverse solution is ompared ith that of the alibration solution, and the orretness of the laser plane an be verified by the standard deviation of the to sets of data in three diretions. Comparison of the to experimental results is as shon in Fig. 6, here the standard deviation of the X diretion as.2 mm, the diretion as.11 mm, and the Z diretion as 1.13 mm.

5 Measurement (mm) Deviation Number of Times -1.5 Fig. 6. Verifiation of plane equations and alibration. The red points are solved by external parameter matrix and blue points are solved by the amera model and laser plane equation. To verify the system measurement auray, a flat glass surfae is used. The glass has a distane of ~2.5 m to the amera. We seleted points on the all as a referene, ith the vertial and horizontal distane beteen them being 3 mm. By measuring the distane beteen horizontal and vertial markers on the glass surfae, the results sho that, the average distane measured as mm, and standard deviation as.87 mm. (d) Fig. 7. Auray evaluation of the proposed system. (a) Flat glass all; (b) Manual measurements beteen to points (3 mm); () Distane beteen points in point loud; (d) Deviation hart. To make the point loud ith olor information, a relay and a single hip miroomputer is developed to synhronize the amera, turntable, and laser. The system an take an image ith laser projetion and the another image ithout laser projetion in the same positions. Image subtration obtains the laser stripe pattern. Next, the laser stripe extration algorithm an be adopted to extrat the stripe enter. This approah improves the robustness of laser strip extration algorithm, and also provides olor information for the laser strip points. An experiment is shon in Fig. 8. Fig. 8a shos to objets ith olor texture, 8b shos the reonstruted 3D point louds, and 8 shos the olor 3D point louds. (a) (b) (a) Number of Times (b) ()

6 () Fig. 8. Reonstrution of the sene (a) ith () and ithout olor (b). By plaing the sanning system in the enter of a room, a full sanning is operated and the dense 3D point loud is obtained as shon by Fig. 9. The size of the room is 382 mm 615 mm. From the reonstrution results, the measured size of the room is about 3818 mm 6141 mm. The top vie of the reonstruted room sene is as shon by Fig. 9a. To side vies ith and ithout olor are displayed by Fig. 9b. A part of the room sene is enlarged and displayed in Fig. 9. () Fig. 9. 3D Reonstrution of indoor senes. (a) Top vie of reonstruted room model; (b) Side vies of the reonstruted room model ith and ithout olor; () A lose observation of part of the reonstruted room sene. (a) IV. CONCLUSION AND FUTURE WORK This paper presents an automati and aurate 3D sanning system for the reonstrution of indoor senes. The system orks based on the laser line sanning priniple. By the proposed alibration method, both amera and laser plane an be preisely alibrated. To align multiple laser profiles, a novel registration method is introdued. By the proposed laser strip detetion method, both a point oordinate and its olor information an be obtained. In the experiments, the measurement error an be ontrolled ithin 1 mm at a orking distane of 2.5 m. With a full sanning proedure, the room sene an be reonstruted automatially ith high preision. Suh a system an be idely used for the indoor sene reonstrution for different appliations like SLAM, VR/AR et. Future ork an address ho to expand the sanning range by the usage of fisheye ameras, hih ill broaden the sanning range and get a more omplete 3D room model. (b) REFERENCES [1] W. Shen and W. Zeng, "Researh of VR modelling tehnology based on VRML and 3DSMAX," Proeedings of 211 International Conferene on Computer Siene and Netork Tehnology, vol. 1, pp , De [2] Sprenger, Bernhard, "Dynamial monitoring and modelling of a oordinate measuring mahine." U.S. Patent No. 9,593, Mar [3]. Nie and Z. Song, "A novel photometri stereo method ith nonisotropi point light soures," rd International Conferene on Pattern Reognition (ICPR), pp , De [4] J. Liao, B. Buhholz, J. M. Thiery, P. Bauszat and E. Eisemann, "Indoor Sene Reonstrution Using Near-Light Photometri Stereo," in IEEE Transations on Image Proessing, vol. 26, no. 3, pp , Marh 217. [5] N. Q. Ann, M. S. H. Ahmad, L. Bayuaji, M. R. Daud and D. Pebrianti, "Study on 3D sene reonstrution in robot navigation using stereo vision," 216 IEEE International Conferene on Automati Control and Intelligent Systems (I2CACIS), pp , Ot. 216.

7 [6] M. okozuka, K. Tomita, O. Matsumoto and A. Banno, "Aurate depth-map refinement by per-pixel plane fitting for stereo vision," rd International Conferene on Pattern Reognition (ICPR), pp , De, 216. [7] J. Gallego and M. Pardàs, "Robust 3D SFS reonstrution based on reliability maps," 214 IEEE International Conferene on Image Proessing (ICIP), pp , Ot [8] E. Turner and A. Zakhor, "Automati Indoor 3D Surfae Reonstrution ith Segmented Building and Objet Elements," 215 International Conferene on 3D Vision, pp , Ot [9] Z. Song. "Use of strutured light for three-dimensional reonstrution." The Chinese University of Hong Kong (People's Republi of China), 28. [1] J. W. Li, et al, "Indoor 3D sene reonstrution for mobile robots using Mirosoft Kinet sensor," th Chinese Control Conferene (CCC), pp , July, 216. [11] Runyang Zou, Xueshi Ge and Geng Wang, "Appliations of RGB-D data for 3D reonstrution in the indoor environment," 216 IEEE Chinese Guidane, Navigation and Control Conferene (CGNCC), pp , Aug [12] X. Chai, F. Wen, X. Cao and K. uan, "A fast 3D surfae reonstrution method for spraying robot ith time-of-flight amera," 213 IEEE International Conferene on Mehatronis and Automation, pp , Aug [13] F. Zhou, and G. Zhang. "Complete alibration of a strutured light stripe vision sensor through planar target of unknon orientations." Image & Vision Computing, vol.23, no. 1, pp , 25. [14] Lenz, Reimar, and D. Fritsh. "Auray of videometry ith CCD sensors." Isprs Journal of Photogrammetry & Remote Sensing vol. 45, no. 2, pp. 9-11, 199. [15] Ghosh, P. K, and S. P. Mudur. "Three-Dimensional Computer Vision: A Geometri Viepoint." Computer Journal vol. 38, no. 1, pp , [16] Pollefeys, Mar, R. Koh, and L. V. Gool. "Self-Calibration and Metri Reonstrution in Spite of Varying and Unknon Internal Camera Parameters. International Journal of Computer Vision vol. 32, no. 1, pp. 7-25, [17] Z. Zhang, "A flexible ne tehnique for amera alibration," in IEEE Transations on Pattern Analysis and Mahine Intelligene, vol. 22, no. 11, pp , Nov 2. [18] Abu-Nabah, Bassam A., A. O. Elsoussi, and E. R. K. A. Alami. "Simple laser vision sensor alibration for surfae profiling appliations." Optis & Lasers in Engineering vol. 84, pp , 216. [19] F. Zhou. "Calibrating Strutured-light Vision Sensor ith Onedimensional Target." Journal of Mehanial Engineering vol. 46, no.18, pp.7-12, 21. [2] Chen, F. Tian, M. A. Zi, and W. U. Xiang. "Calibration of light plane in line strutured light sensor based on ative vision." Guangxue Jingmi Gongheng/optis & Preision Engineering vol.2, no. 2, pp , 212. [21] J. Fan, F. Jing, Z. Fang and Z. Liang, "A simple alibration method of strutured light plane parameters for elding robots," th Chinese Control Conferene (CCC), pp , July 216.