EGISTATION OF TEESTIAL LASE SCANNE DATA USING IMAGEY Khall Al-Manasr, Ph.D student Clve S. Fraser, Professor Department of Geomats Unversty of Melbourne Vtora 3010 Australa k.al-manasr@pgrad.unmelb.edu.au.fraser@unmelb.edu.au ABSTACT Buldng 3D models usng terrestral laser sanner (TLS) data s urrently an atve area of researh, espeally n the felds of hertage reordng and ste doumentaton. Multple TLS sans are often requred to generate an oluson-free 3D model n stuatons where the objet to be reorded has a omplex geometry. The frst task assoated wth buldng 3D models from laser sanner data n suh ases s to transform the data from the sanner s loal oordnate system nto a unform Cartesan referene datum, whh requres suffent overlap between the sans. Many TLS systems are now suppled wth an SL-type dgtal amera, suh that the sene to be sanned an also be photographed. The provson of overlappng magery offers an alternatve, photogrammetr means to aheve pont loud regstraton between adjaent sans. The mages from the dgtal amera mounted on top of the laser sanner are used to frst relatvely orent the network of mages, and then to transfer ths orentaton to the TLS statons to provde exteror orentaton. The proposed approah, alled the IB method for Image-Based egstraton, offers a one-step regstraton of the pont louds from eah sanner poston. In the ase of multple sans, exteror orentaton s smultaneously determned for all TLS statons by bundle adjustment. Ths paper outlnes the IB method and dsusses test results obtaned wth the approah. It wll be shown that the photogrammetr orentaton proess for TLS pont loud regstraton s effent and aurate, and offers a vable alternatve to other approahes, suh as the well-known teratve losest pont algorthm. INTODUCTION The frst requrement of 3D modelng an objet or sene va terrestral laser sannng (TLS) s to transform the overlappng pont louds from adjaent sans nto a unform Cartesan referene oordnate system. One of the popular approahes adopted for ths regstraton proess s the Iteratve Closest Pont (ICP) algorthm for surfae mathng developed by Besl & MaKay (1992). Several varatons of the ICP algorthm have been formulated (Chen & Medon, 1992; Zhang, 1994; Masuda & Yokoya, 1995; Bergevn et al., 1996), nludng the use of sensor aquston geometry to searh for pont orrespondenes, as proposed by Park & Subbarao (2003). Also, an ntal soluton for a subsequent applaton of the ICP method usng the normal dstrbuton transform (NDT) has been reported by pperda & Brenner (2005), whle Gruen & Aka (2005) have suggested the Least Squares 3D Surfae Mathng (LS3D) method. For a revew of urrent 3D surfae regstraton algorthms the reader s referred to Campbell & Flynn, (2001) and Gruen & Aka (2005). Terrestral laser sanner manufaturers generally supply an SL-type dgtal amera wth the laser sanner to faltate generaton of photo-realst 3D vrtual models. In ths paper, a pont loud regstraton method s desrbed, whh makes use of the magery aompanyng the TLS pont loud to determne sanner exteror orentaton and thus san-to-san regstraton. The dgtal amera s frst photogrammetrally albrated va a self-albraton proess, after whh t s mounted on the sanner. The amera s exteror orentaton wth respet to the TLS referene oordnate system s then determned, agan photogrammetrally. Subsequently, the photogrammetr orentaton of magery from a dgtal amera mounted on the TLS s performed va relatve orentaton or bundle adjustment n order to transform all the sanned data nto a ommon oordnate system. ASPS 2006 Annual Conferene
OVEVIEW OF THE EGISTATION METHOD A Nkon D100 6-megapxel dgtal amera wth an 18mm lens s shown rgdly mounted on a egl LMS-Z210 TLS n Fg. 1. If the exteror orentaton parameters of the amera oordnate system (x,y,z) wth respet to the laser oordnate system (X,Y,Z) are known, then the relatve orentaton between overlappng mages from two statons reorded wth the metrally albrated amera an also provde the transformaton parameters requred to o-regster the two aompanyng laser pont louds. Fgure 1. The Nkon D100 amera rgdly attahed to the egl TLS. Followng a self-albraton stage, the amera poston wth respet to the laser sanner oordnate system need only be reovered one. TLS and mage data from two or more statons are then reorded, wth there beng a requrement for mage overlap, but not neessarly overlap between the TLS pont louds. A photogrammetr orentaton for the two or more overlappng mages provdes the exteror orentaton parameters for the amera statons wth respet to a sngle referene oordnate system. By also knowng the amera poston wthn the TLS oordnate system, the regstraton of pont louds s dretly establshed. A prmary motvaton for ths mage-based regstraton (IB) proess s avodane of the often problemat proess of dentfyng onjugate ponts wthn the laser san data. THE MATHEMATICAL MODEL 3D Coordnate Transformaton The mathematal relatonshp between two overlappng TLS pont louds an be smply expressed as a 3D smlarty transformaton: X X X 1 2 = Y λ Y Y 1 2 Z Z Z 1 2 (1) where (X 1, Y 1, Z 1 ) and (X 2, Y 2, Z 2 ) represent San 1 and San 2 oordnates; a rotaton matrx, whh s formed from three axal rotaton angles here termed ω, φ and к; and (X, Y, Z ) the translaton omponents between San 1 and San 2. Sne the laser range data establshes absolute sale, the applable sale fator λ has unt value, so Eq. 1 represents a rgd body transformaton. One the sx transformaton parameters (sale s assumed fxed) are omputed between the pont louds, the XYZ oordnates of all san ponts an be transformed nto a ommon oordnate system. Camera Calbraton The photogrammetr regstraton approah ntally nvolves a one-off amera albraton va an ndependent self-albraton network. Wth the use of approprate oded targets and software, the mage measurement and omputaton of albraton parameters s a fully automat proess, as desrbed for example n Cronk et al. (2006). The software system used for ths stage of the projet was Australs (Fraser & Edmundson, 2000; Photometrx, ASPS 2006 Annual Conferene
2006). The automaton aspet s of mportane for amera albraton beause t s wse to perodally re-hek the amera nteror orentaton parameters. The albraton model s that of the extended ollnearty equatons, wth eght addtonal parameters: where and 1 x x +Δ x = 0 3 2 y y +Δ y = 0 3 1 2 3 = X Y Z X Y Z x 2 4 6 2 2 Δ x = x Δ + xr K + xr K + xr K + (2 x + r ) P+ 2Pxy 0 1 2 3 1 2 y 2 4 6 2 2 Δ y= y Δ + yr K + yr K + yr K + 2 Pxy+ (2 y + r ) P 0 1 2 3 1 2 (2) (3) Here, (x,y) are mage oordnates, x and y departures from ollnearty; a orreton to the prnpal dstane ; x 0 and y 0 the prnpal pont oordnates; X,Y,Z objet pont oordnates; X, Y and Z translatons; K the oeffents of radal lens dstorton and P those of deentrng dstorton. Ths physal addtonal parameter model s urrently preferred for dgtal amera self-albraton (eg Fraser, 1997). For pratal reasons, the amera s removed from the TLS for self-albraton, sne amera roll s preluded whle t s mounted, yet roll angle dversty s rtal for an aurate reovery of nteror orentaton parameters. Camera to TLS Orentaton Followng the albraton phase, the amera s agan mounted on the TLS at a fxed algnment, as n Fg. 1, and an mage and a san are reorded. Ths one-off operaton allows the poston and orentaton of the amera wth respet to the TLS oordnate system to be determned usng spatal reseton. The mathematal model for spatal reseton s the well-known formulaton based on the ollnearty equatons. It s effetvely that of Eq. 2 wth (x,y) onsttutng the observatons, and the parameters of exteror orentaton (the three rotaton angles formng and X, Y and Z ) beng the unknowns. Intal startng values for the least-squares soluton of the reseton problem are determned by a losed-form algorthm smlar to that of Quan & Lan (1999) and Zeng & Wang (1992). The ontrol ponts used, whh should number 10 or more well spread ponts for auray and relablty, are seleted from the TLS san data. elatve Orentaton of Images Wthn the TLS survey, two or more sans are reorded, suh that the mages from the TLS statons suffently overlap to support a photogrammetr relatve orentaton or bundle adjustment. The well-known oplanarty model (eg Mkhal et al., 2001) has been adopted to solve the relatve orentaton of an mage par, sne approxmated objet spae oordnates are not requred for a soluton. The oplanarty model an be formulated from the perspetve projeton represented by Eq. 2, for the ase of one mage beng relatvely orented to a seond, as: 0 bz by Τ Τ u 0 = 0 1 bz bx u (4) 2 2 0 by bx where T u = x' y' ( ) ASPS 2006 Annual Conferene
Here, (x,y ) represent mage oordnates refned and orreted for prnpal pont offset and lens dstorton, the matrx 2 desrbes the rotaton of the seond mage wth respet to the frst, and by and bz are translatons. The translaton bx, whh les n the baselne, an be assgned an arbtrary value sne sale annot be reovered from the oplanarty model. Sale s appled to the relatvely orented network usng pont-to-pont dstanes determned between mage dentfable laser sanned ponts. The oplanarty equatons are solved va least-squares, wth a mnmum of fve sutably dstrbuted ponts beng neessary to reover the three rotaton angles and two translatons, though use of 10 or more orrespondng ponts s reommended. As n the earler reseton, the determnaton of ntal values n the least-squares soluton s rtal f the proess s to be hghly automated. To solve the problem of ntal value determnaton, a Monte Carlo approah desrbed n Cronk et al. (2006) has been adopted. In ths approah, the most plausble ntal relatve orentaton solutons are determned, effetvely by tral and error. These are then refned by least squares and the orret soluton s automatally determned. In the ase of multple overlappng mages and TLS data sets, t s possble to further refne the exteror orentaton n a bundle adjustment, though ths well known model wll not be further desrbed n detal here. Pont Cloud egstraton The mathematal relatonshp between the amera and TLS oordnate system an be expressed as follows: x y z = λ C X X Y Y Z Z (5) where (x, y, z) represent the amera s oordnate system and (X, Y, Z) are the TLS pont oordnates. The rotaton matrx C expresses the relatve algnment between the axes of the two systems and (X, Y, Z ) desrbes the poston of the amera wth respet to the orgn of the TLS oordnate system. In ths ase the sale fator λ agan has unt value. Wth the relatve orentaton between two mages and j determned, the pont loud regstraton an be establshed va the transformaton establshed from a ombnaton of Eqs. 4 and 5: where X Y Z = 1 A. 1 C. 1 j,. x y z j bx + by bz + X Y Z (6) x y z j = C. A j X. Y Z j X Y Z and s the referene pont loud wth j beng the data set whose oordnates are to be transformed. The amera poston and orentaton n the TLS oordnate system at a spef algnment of the sanner, when the rotaton angle A of the sanner shown n Fg. 2 s zero, are expressed by the 3x3 rotaton matrx C and the translaton vetor (X, Y, Z ). The 3x3 rotaton matrx j, and the translaton vetor (bx, by, bz) are formed by the exteror orentaton of amera staton j wthn the referene oordnate system of amera staton ; and A s a 3x3 rotaton matrx defnng the TLS rotaton around ts Z-axs desrbed by the angle A at the tme of exposure. EXPEIMENTAL ESULTS Three expermental applatons of the IB method are now summarzed. The frst was a 3-san survey of amera albraton testfeld omprsng an array of targets of known oordnates, the seond nvolved the 3D ASPS 2006 Annual Conferene
modelng of a ultural hertage ste, and the thrd was seleted to llustrate the regstraton of non-overlappng TLS pont louds usng the IB approah. Fgure 2. elatonshp between the TLS oordnate system and the optal axs of the amera. Survey of a Calbraton Testfeld The frst experment nvolved the modelng of a target feld used for researh nto sensor albraton n lose range photogrammetry. The testfeld omprses a mostly planar array of 160 retro-refletve targets, mounted upon both a wall and a seton of elng, as shown n Fg. 3. Ths testfeld represented an deal ste at whh to evaluate the automat relatve orentaton and regstraton of overlappng pont louds. The nne large square targets were used to ompute the amera poston and orentaton wthn the TLS oordnate system (Eq. 5). Fgure 3. The albraton wall omprsed of retro-refletve targets. A photogrammetr network of 14 onvergent mages was establshed to self-albrate the amera. The mages were reorded from a set-bak dstane of about 4m. The oded targets n Fg. 3 (small squares) were used to faltate automat network orentaton and self-albraton va the Australs software system. The hgh qualty of the albraton s ndated by the low magntude of the mage oordnate resduals, whh had an MS value of 0.05 pxels or 0.4 m. The Nkon D100 was mounted on the egl TLS, as per Fg. 1, followng the amera albraton. In the next stage of the proess the relatonshp between the amera and TLS oordnate systems was establshed. A sngle san of the albraton wall was reorded, along wth a sngle mage. The entrods of the nne promnent retro-targets seen n Fg. 3 were measured n the mage and a spatal reseton performed. The resultng auray of the amera poston and orentaton was 0.5mm for poston and 10 seonds of ar for the three orentaton angles. Gven the nherent auray of the egl sanner, of about 25mm over a 4m dstane, the reseton result was deemed satsfatory for the subsequent regstraton and was expeted to produe a regstraton auray that would surpass that usng TLS san data alone. The IB method was performed to regster the three overlappng sans of the albraton wall, as shown n Fg. 4. Frstly, mages assoated wth San 3 and 2 were relatvely orented va Eq. 4, usng more than 25 automatally extrated mage ponts. After the reseton of the mage assoated wth San 1, usng the 3D objet pont oordnates whh were determned from the relatve orentaton proess of mages 2 and 3, a bundle adjustment was performed usng all three mages. ASPS 2006 Annual Conferene
The transformaton of the TLS ponts n Sans 1 and 2 nto San 3 was performed usng Eq. 6, thus ompletng the regstraton proess. Beause the retro-targets ould be dentfed by ther ntensty haratersts, t was possble to ompare the fnal XYZ oordnates to known values whh had prevously been determned n a photogrammetr survey of sgnfantly hgher auray. The resultng resduals aganst the 100 hek ponts are shown n Table 1. The 2mm auray aheved s vewed as qute aeptable and well wthn that antpated for the TLS. A further regstraton of the three TLS pont louds was reated usng the ICP method, n whh the retrotargets were dentfed by ther ntensty values and ompared to the known hek ponts. As shown n Table 1, ths regstraton yelded an auray of 2.5mm. Fgure 4. Three-san overage of the albraton wall. Table 1. MSE of regstered pont louds wth respet to albraton wall ontrol ponts. egstraton method No. of ontrol ponts Overall MSE (mm) ICP 100 2.5 IB method 100 2.0 Modelng of Captan Cook s Cottage The seond experment nvolved the 3D modelng of Captan Cook s Cottage, a hertage ste and popular tourst attraton n Melbourne. The ottage was sanned from seven dfferent statons wth the egl sanner. The laser data omprsed about 3.5 mllon ponts. Seven mages were reorded wth the mounted Nkon D100 amera, one from eah TLS staton. Suffent overlap was provded between mages so as to ensure robust relatve orentaton and subsequent bundle adjustment. All mages were reorded wth the laser sanner ponted n ts X-dreton so A = I as ndated n Fg. 5. Fgure 5. Sanner/amera staton geometry for the survey of Cook s ottage. ASPS 2006 Annual Conferene
In the photogrammetr soluton, the mage n San 1 was adopted as the referene san for the network, and the subsequent bundle adjustment of all seven mages was then performed. The mage measurements were manual sne no speal targets were plaed on the objet. The mages assoated wth San 1 and 2 were frst relatvely orented va Eq. 4 usng more than 30 manually extrated mage ponts. Care was taken to selet ponts suh that at least four, the mnmum requred for the losedform spatal reseton algorthm, would be dentfable and measurable n the mages from the statons San 3 though to San 7. A bundle adjustment of all mages was then performed, whh produed an MS value of mage oordnates of 0.5 pxels. The resultng 1-sgma auray of the photogrammetrally determned objet ponts was at the 2mm level, e well wthn the spefed auray of the egl TLS. egstraton of the laser sans was then arred out va Eq. 6, wth the resultng, regstered 3D pont loud beng shown n Fg. 6. Fgure 6. The 3D model of Cooks' ottage from TLS data. The resultng MS dsrepany between the model regstered va the IB approah and that of an ICPgenerated model was 2.7mm, wth the largest resduals beng 4mm n the grassed area n front of the ottage (see Fg. 7). A vsual analyss of the error map does not suggest the presene of any systemat msalgnment between the two 3D models, whh ndated that for all pratal purposes the two solutons were equvalent. Fgure 7. Map of dsrepanes between fnal losest ponts n the IB & ICP data sets (unts are m). ASPS 2006 Annual Conferene
Modelng a Buldng Entrane The thrd experment was arred out to demonstrate the ablty of the proposed IB method to regster nonoverlappng TLS pont louds, as llustrated n Fg. 8. Two sans were reorded wth the egl sanner, along wth two mages from the Nkon D100 amera. The amera was pontng to an adjaent buldng next to the ste beng sanned. Ths ensured that there would be suffent overlap between mages to support robust relatve orentaton and subsequent bundle adjustment but no overlap between the TLS pont louds. Another two mages were reorded (shown as Image 1 and Image 2) usng the demounted Nkon D100, n order to mprove the strength of the photogrammetr network. Fgure 8. Sanner/amera staton geometry hghlghtng non-overlappng TLS data. A relatve orentaton was performed between the mages from the statons San 1 and San 2, followed by a 4- mage bundle adjustment whh produed an MS value of mage oordnate resduals of 0.4 pxels. The resultng 1-sgma auray of the photogrammetrally determned objet ponts was 2mm. egstraton of the two laser sans was then arred out va the IB method (Eq. 6). In order to hek the auray of the regstered 3D pont loud from the IB approah, nne ontrol ponts, whh had prevously been determned n a photogrammetr survey of sgnfantly hgher auray, were dentfed manually n the regstered 3D pont loud. The resultng absolute MS pont postonng errors from the IB approah are shown n Table 2, where X & Y are planmetr oordnates. Table 2: Pont postonng errors n the IB regstraton. Number of hek ponts MSE X(mm) MSE Y(mm) MSE Z(mm) 9 4.2 4.5 3.1 CONCLUSIONS The IB method for regstraton of laser sanner data usng photogrammetr orentaton has been desrbed, and test results from three tral applatons of the method have been reported. The IB method has been shown to provde a pratal alternatve to urrent TLS regstraton approahes suh as the ICP algorthm. Moreover, the method s fast ompared to the ICP algorthm, and has the prospet of beng more aurate, though ths aspet needs further onfrmaton. One beneft of the IB method s that the seleton of onjugate ponts n dgtal magery s generally a more straghtforward and less error prone proess than the same operaton wth laser san data that nludes ntensty values. The proposed method s robust and does not requre any overlap between the separate TLS pont louds. Ths an be a sgnfant beneft. Also, the IB approah an overome known dffultes assoated wth exstng laser sanner regstraton algorthms and n many nstanes t an serve as an alternatve to the ICP algorthm and LS3D approah. The IB method an also onsttute an ntal soluton for both the ICP and the LS3D approahes where regstraton va these algorthms mght be more approprate, for example where t s only possble to form a weak relatvely orented network of mages. ASPS 2006 Annual Conferene
EFEENCES Bergevn,., Souy, M., Gagnon, H. and Laurendeau, D. (1996). Towards a general mult-vew regstraton tehnque. IEEE Transatons on Pattern Analyss and Mahne Intellgene, 18(5), 540 547. Besl, P.J. and MKay, N.D. (1992). A method for regstraton of 3-D shapes. IEEE Transatons on Pattern Analyss and Mahne Intellgene, 14(2): 239-256. Campbell,.J. and Flynn, P.J. (2001). A survey of free-form objet representaton and reognton tehnques. Computer Vson and Image Understandng, 81(2): 166-210. Chen, Y. and Medon, G. (1992). Objet modellng by regstraton of multple range mages. Image and Vson Computng, 10(3): 145-155. Cronk, S., Fraser, C.S. and Hanley, H. (2006). Automat albraton of olour dgtal ameras. Submtted to Photogrammetr eord, 19 pages. Fraser, C.S. (1997). Dgtal amera self-albraton. ISPS Journal of Photogrammetry and emote Sensng, 52(4):149-159. Fraser, C.S. and Edmundson, K.L. (2000). Desgn and mplementaton of a omputatonal proessng system for off-lne dgtal lose-range photogrammetry. ISPS Journal of Photogrammetry and emote Sensng, 55(2): 94-104. Gruen, A. and Aka, D. (2005). Least squares 3D surfae and urve mathng. ISPS Journal of Photogrammetry and emote Sensng, 59(3): 151-174. Masuda, T. and Yokoya, N. (1995). A robust method for regstraton and segmentaton of multple range mages. Computer Vson and Image Understandng, 61(3): 295-307. Mkhal, E.M., Bethel, J.S. and MGlone, J.C. (2001). Introduton to Modern Photogrammetry. John Wley & Sons, New York. 479 pp. Park, S.Y. and Subbarao, M. (2003). A fast pont-to-tangent plane tehnque for mult-vew regstraton. IEEE Internatonal Conferene on 3-D Dgtal Imagng and Modelng, Banff, Otober 6-10, pp. 276-283. Photometrx (2006). Webste for Australs system: www.photometrx.om.au, aessed 30 January 2006. Quan, L. and Lan Z. (1999). Lnear N-pont amera pose determnaton. IEEE Transatons on Pattern Analyss and Mahne Intellgene, 21(8): 774-780. pperda, N. and Brenner, C. (2005). Marker-Free regstraton of terrestral laser sans usng the normal dstrbuton transform. ISPS Workshop on Vrtual eonstruton and Vsualzaton of Complex Arhtetures, Mestre-Vene, Italy, August 22-24, on CD-OM. Zeng, Z. and X. Wang (1992). A general soluton of a losed-form spae reseton. Photogrammetr Engneerng and emote Sensng, 58(3): 327-338. Zhang, Z. (1994). Iteratve pont mathng for regstraton of free-form urves and surfaes. Internatonal Journal of Computer Vson, 13(2):119-152. ASPS 2006 Annual Conferene