Simultaneous Intrinsic and Extrinsic Parameter Identification of a Hand-Mounted Laser-Vision Sensor

Transcription

1 Sensors,, ; do:.339/s9875 OPEN ACCESS sensors ISSN Artcle Smultaneous Intrnsc and Extrnsc Parameter Identfcaton of a Hand-Mounted Laser-Vson Sensor Jong Kwang Lee, Kho Km, Yongseok Lee and Takyeong Jeong, * Fuel Cycle System Engneerng Technology Development Dvson, Korea Atomc Energy Research Insttute, Daejeon, , Korea; E-Mals: leejk@kaer.re.kr (J.K.L.); khkm5@kaer.re.kr (K.K.) Department of Electronc Engneerng, Myongj Unversty, Yongn-cty, , Korea; E-Mal: leeys5867@mju.ac.kr (Y.L.) * Author to whom correspondence should be addressed; E-Mal: ttjeong@mju.ac.kr; Tel.: ; Fax: Receved: 8 August ; n revsed form: 5 September / Accepted: 6 September / Publshed: 9 September Abstract: In ths paper, we propose a smultaneous ntrnsc and extrnsc parameter dentfcaton of a hand-mounted laser-vson sensor (HMLVS). A laser-vson sensor (LVS), consstng of a camera and a laser strpe projector, s used as a sensor component of the robotc measurement system, and t measures the range data wth respect to the robot base frame usng the robot forward knematcs and the optcal trangulaton prncple. For the optmal estmaton of the model parameters, we appled two optmzaton technques: a nonlnear least square optmzer and a partcle swarm optmzer. Best-ft parameters, ncludng both the ntrnsc and extrnsc parameters of the HMLVS, are smultaneously obtaned based on the least-squares crteron. From the smulaton and expermental results, t s shown that the parameter dentfcaton problem consdered was characterzed by a hghly multmodal landscape; thus, the global optmzaton technque such as a partcle swarm optmzaton can be a promsng tool to dentfy the model parameters for a HMLVS, whle the nonlnear least square optmzer often faled to fnd an optmal soluton even when the ntal canddate solutons were selected close to the true optmum. The proposed optmzaton method does not requre good ntal guesses of the system parameters to converge at a very stable soluton and t could be appled to a knematcally dssmlar robot system wthout loss of generalty.

2 Sensors, 875 Keywords: hand-mounted laser-vson sensor; parameter dentfcaton; partcle swarm optmzaton. Introducton To measure the range data of objects under an unknown workng envronment, a range sensng devce has been wdely appled to varous robotc applcatons [-6]. In these researches, the range sensng devce s usually nstalled on the robot hand and t s equpped wth varous sensors such as a camera(s), laser-vson sensor(s) and/or sonar(s). Snce parameter dentfcaton, also referred to as calbraton, s crucal to the system accuracy, t s consdered as an mportant step before performng any measurement task. As for the robotc measurement system whch ntegrates a robot wth a range sensng devce(s), generally, four dfferent calbraton procedures should be performed: sensor calbraton, hand-to-sensor calbraton, robot calbraton, and base calbraton [7]. In ths work, we concentrated on the frst two calbratons by assumng that we know the poston of the calbraton ponts exactly and that the geometrc lnk parameter errors of the robot manpulator are neglgble. A laser-vson sensor (LVS) consstng of a CCD camera(s) and a laser strpe projector has been frequently used as an actve rangng devce [-9] and a feature detecton sensor []. It s mathematcally modeled based on an optcal trangulaton prncple [] or a converson matrx [3] whch defnes the geometrcal relatonshp between the slt beam coordnates and ther correspondng mage coordnates. As for the model parameters of the LVS, there exst two parameter sets to be dentfed: the ntrnsc parameters and the extrnsc parameters. As for a camera, the ntrnsc parameters model the nternal geometry and optcal characterstcs of the mage sensor whch determne how lght s projected through the lens onto the mage plane of the sensor. They consst of the focal length, the lens dstorton coeffcents, the optcal center, and the magnfcaton coeffcent of the CCD cell []. As the extenson of the ntrnsc parameters of the camera, we consdered the ntrnsc parameters of the LVS whch consst of the ntrnsc camera parameters as well as laser strpe generator parameters such as a baselne dstance and a projecton angle wth respect to a camera coordnate frame. The extrnsc parameters of the LVS are related to the poston and orentaton of the camera wth respect to the robot hand coordnate system. Most approaches to the calbraton of a hand-mounted LVS have made use of the mult-stage technque [-6], that s, the camera and laser parameter calbraton stages were performed separately. The extrnsc parameters of the omndrectonal laser-vson sensor used n a free-rangng robot were dentfed after solvng the ntrnsc parameters based on the exstng camera calbraton method [3]. On the other hand, the extrnsc parameters were dentfed frst to estmate the orentaton and poston of a camera wth respect to a laser range fnder and then the camera ntrnsc calbraton was performed [5]. The mult-stage technque, however, s known to have drawbacks such as error propagaton. Recently, the onlne re-calbraton of a LVS whch s mounted on a Cartesan carrage has been proposed to acheve good resoluton and to avod occlusons when the sensor geometry s modfed onlne [8,9]. In the method, sensor parameters are determned by usng the Bezer network wthout any calbraton reference. In the robotc measurement system, the poston and orentaton of the sensor could be changed by manpulatng a robot arm to avod occluson problems. Another approach s

3 Sensors, 8753 self-calbraton, whch ams to mprove a practcal mplementaton by ntroducng physcal constrants such as a fxed pont, a straght lne, a crcle, or a sphere to the system [,4]. In these works, they used a commercal LVS, thus, the ntrnsc parameters of the LVS were assumed to be known as a pror. However, t s analyzed n ths paper that small amount of nexactness of model parameters could have a consderable effect on the measurement errors. Ths paper s organzed as follows. In Secton, we address the hand-mounted laser-vson sensor model and analyze ts measurement range and resoluton. The parameter dentfcaton based on a partcle swarm optmzaton s proposed n Secton 3, and ts performance s compared wth conventonal non-lnear least square optmzaton n Secton 4. Secton 5 llustrates expermental results. The concluson s gven n the fnal secton.. Hand-Mounted Laser-Vson Sensor Model Fgure shows a schematc model for the hand-mounted laser-vson sensor (HMLVS). A 3D pont P(, x yz,) n the camera coordnate system s transformed nto an undstorted mage coordnate (X u, Y u ) by usng a perspectve projecton wth a pnhole camera geometry. Snce the pnhole model s only an approxmaton of the real camera projecton, a nonlnear lens dstorton s consdered to mprove the measurement accuracy [,,]. The dstorted or true mage coordnate (X d, Y d ) s corrected by usng the followng equaton: x X f X k r u = = d /( + ) z = y Y f Y k r u = = d /( + ) z = where r = X d + Y ; f s the effectve focal length of the camera; k d represents the coeffcents of the radal lens dstorton seres. Snce a suffcent accuracy can be acheved wth a frst-order dstorton, we neglect the hgh order coeffcents and use k = k. The coordnate (X u, Y u ) on the mage plane s transformed to a D mage pxel (X f, Y f ) n a computer frame memory by usng the magnfcaton coeffcents (S u, S v ) and a center of the computer frame memory (C u, C v ) as: X Y f f X u C = Su Yu Cv = S u Next, the 3D poston of a pont s computed through an optcal trangulaton prncple. As shown n Fgure, a laser strpe generator emts a plane of a beam wth an angle θ relatve to the Z C axs. The pont P (, x yz,) on the object surface s projected onto the dgtzed mage at the pxel (X f, Y f ) and controlled by the effectve focal length of the lens f and the baselne dstance, H. Accordngly, the LVS can obtan 3D nformaton n the camera coordnate system through measurng the mage pxel coordnate (X f, Y f ) whch corresponds to the 3D coordnates P(, x yz,) of the llumnated laser pont as: u () ()

4 Sensors, 8754 C x X f P = y = ρ Y = ρu (3) f z f where ρ = Y f H. + f tanθ Fgure. The laser-vson sensor model. The laser-vson sensor conssts of a camera and a laser strpe generator. A 3D range data s obtaned by usng both the camera projecton model and the optcal trangulaton prncple. (a) Laser-vson sensor. (b) Laser-vson sensor geometry. y c Laser strpe generator y c z c H ( Xu, Y u) P( x, yz, ) V Object surface Camera frame x c f U θ z c Image plane ( X, Y ) d d (a) (b) The ntrnsc parameters of the LVS model nclude the ntrnsc camera parameters, f, S, S, C, C, k, as well as the mountng parameters of the laser strpe generator wth respect to { u v u v } the camera coordnate frame, { H, θ }. Because the LVS s nstalled on the last lnk of the robot manpulator, addtonal extrnsc parameters of the LVS, whch defne the poston and the orentaton of the camera frame wth respect to the robot hand frame, should be consdered. A knematc model of a robot manpulator can be modeled by usng the Denavt-Hartenberg conventon. Let B HT be a 4 4 homogeneous transform matrx of a robot manpulator wth n degree of freedom between the base frame and the hand frame, that s: R P T T T T (4) B n- N N H = n = where +T s the homogeneous transformaton matrx between two consecutve coordnate frames and +. If we denote the homogeneous transformaton matrx between the hand frame and the camera frame as H T : C

5 Sensors, 8755 H C r r r3 tx r r r3 t y RC PC T = =, r3 r3 r33 t z (5) where t x, t y and t z denote the poston of the camera frame relatve to the hand frame; the elements, r j n the rotaton matrx R C can be represented as a functon of the Euler angle Rx ( ω ), R ( φ ), Rz ( ϕ) as: cosω cosφcosϕ snω snϕ cosω cosφsnϕ snω cosϕ cosω snφ R C = snω cosφcosϕ + cosω snϕ snω cosφsnϕ + cosω cosϕ snω snφ (6) snφcosϕ snφsnϕ cosφ Snce the transformaton matrx between the robot base frame and the camera frame, B CT, can be represented as: B = B H, C H C T T T (7) the poston vector of the laser beam reflected by the object surface, B P, s represented relatve to the robot base coordnate frame by usng Equatons (3) and (7) as: B C P B P RN PN RC PC ρu = CT = where C P s the poston vector of the reflected laser beam n the camera coordnate system. Therefore, the poston of the reflected lght wth respect to the robot base frame (as shown n Fgure ) s calculated by usng the followng system model: B P = ρ R R U + R P + P (9) N C N C N. Fgure. Knematc model of the hand-mounted laser-vson sensor (HMLVS). y (8) x C B CT B T H H C T x H y C z C y H z H C P z B y B B P x B

6 Sensors, 8756 The baselne dstance ( H ) and the projecton angle (θ ) affect the measurement range and resoluton of the laser-vson sensor. Here, the resoluton s defned as the dsplacement n the 3D real space per one pxel n the mage plane. To nvestgate these characterstcs, we consder a geometrc relaton as shown n Fgure 3, where a reference plane moves parallel to an mage plane along wth Z c axs. A laser lne llumnated horzontally on the reference plane shfts vertcally as the dstance between the mage plane and the reference plane vares. Accordng to Equaton (3), the baselne dstance acts as a scale factor for the 3D real coordnate of the llumnated laser lne. Therefore, as H ncreases, the measurement range s ncreased by sacrfcng the resoluton as shown n Fgure 4. For a horzontally llumnated laser lne on the reference plane, the resoluton about x c axs on the mage coordnate system, Δx / Δ X, s constant snce ΔY / ΔX s zero. As the reference plane approaches to the mage c f f f plane, the llumnated laser lne moves vertcally upward drecton about V axs on the mage plane. In ths case, the measurement range decreases whle the sensor resoluton ncreases as shown n Fgures 4 and 5. Fgure 3. The geometrc relaton between a 3D llumnated laser pont and ts D mage projecton. y c Reference plane H V U θ z c The baselne dstance and the projecton angle are desgn parameters of the laser-vson sensor, where they should be selected approprately based on the requrements of target applcatons. If we desgn a sensor wth a resoluton of less than mm/pxel, we could choose the baselne dstance of mm and the projecton angle s 5 degree. In ths case, measurement ranges about x c, y c, and z c are 55. mm,.3 mm, and 6.3 mm, respectvely. To further ncrease the sensor resoluton wth the same confguraton, measurement range should be decreased. Ths can be acheved by approachng the sensor to an object by manpulatng a robot arm and ts permssble dstance can be observed by checkng the laser lne n the V axs of the mage plane. To nvestgate the effect of the nexactness of the baselne dstance and the projecton angle on the measurement errors, we carred out smulatons. We assumed % nexactness of the baselne dstance and the projecton angle: the baselne dstance s set as mm for ts real value of 99 mm, and the projecton angle s 5 degree for ts real value of.5 degree. Measurement errors are shown n Fgure 6. It s mportant to note that these errors arsen from the nexactness are larger than the sensor resoluton as shown n Fgures 4 and 5. Moreover, snce the baselne dstance and the projecton angle

7 Sensors, 8757 are measured wth respect to the camera coordnate frame, they should be determned after the orgn of the camera coordnates was obtaned. In addton, mechancal errors arsen from manufacturng and assemblng should be consdered for better accuracy. So, they should be dealt wth unknown parameters. Fgure 4. Measurement range and resoluton for dfferent baselne dstances, where the projecton angle s 5 degree. Measurement range x c mm mm 5 mm mm Resoluton x c mm mm 5 mm mm (a) Measurement range y c mm mm 5 mm mm Resoluton y c mm mm 5 mm mm (b) Measurement range z c mm mm 5 mm mm Resoluton z c mm mm 5 mm mm (c)

8 Sensors, 8758 Fgure 5. Measurement range and resoluton for dfferent projecton angles, where the baselne H s mm. Measurement range x c degree degree 5 degree 3 degree Resoluton x c degree degree 5 degree 3 degree (a) Measurement range y c degree degree 5 degree 3 degree Resoluton y c degree degree 5 degree 3 degree (b) Measurement range z c degree degree 5 degree 3 degree Resoluton z c degree degree 5 degree 3 degree (c) Fgure 6. Measurement errors of the LVS wth % nexact value of (a) the baselne and (b) the projecton angle. 5 4 x c y c z c Measurement error 3 Measurement error x c y c z c (a) (b)

9 Sensors, Parameter Identfcaton 3.. Objectve Functon As a frst step to dentfy the model parameters, we should defne an objectve functon to be optmzed. Let q be a vector consstng of the unknown ntrnsc and extrnsc parameters of the HMLVS, that s: For a notatonal convenence, we rewrte q as: q = f, Su, Sv, Cu, Cv, k, H, θωφϕ,,,, tx, ty, t z () [ q q q ] T q =,,, n () where n s the number of unknown parameters (n ths case, 4). Searchng boundary on parameters s set as: L U q q, q () L U where q and q denote the lower and upper bounds of q respectvely. Any reasonable nterval whch may cover the possble parameter values may be chosen as the bound of parameter q. In ths work, we estmated a best-ft parameter vector q * by mnmzng the summed squared error of m nonlnear functons: m * q = arg mn F( q) = ( f ( q )) (3) q = where f ( q ) = E s a Eucldean norm of the error vector E whch s gven by: 3.. Optmzaton Technques E= P ρr R U R P P (4) N C N C N. We appled two optmzaton technques: nonlnear least squares optmzaton (NLSO) and partcle swarm optmzaton (PSO). In the followng paragraphs, we ntroduce them n bref. NLSO s a popular algorthm that s frequently used to fnd the mnmum of a multvarate functon represented as the sum of squares of the nonlnear functons. Among the varous NLSO algorthms, we used the Levenberg-Marquardt algorthm (LMA) whch s also referred to as the damped Gauss-Newton Method [3]. The LMA starts wth an ntal canddate soluton. Gven a current soluton vector q k, the LMA generates the next soluton vector q k + by usng the followng equaton: where a vector of adjustments for the unknowns, q = q +Δq (5) k+ k k Δq k, s computed as: k f k f k Δ q = ( q ) ( q ) (6) Ths process s repeated untl F( q k ) or Δq k s suffcently small; a maxmum number of teratons are completed. In the LMA, the Hessan matrx s approxmated as: T

10 Sensors, 876 and the gradent s computed as: f ( q ) = J J + λ I (7) T k k k k f q = J q (8) T ( k) k f ( k) where J k s a Jacoban matrx whch contans the frst dervatves of the error vectors. The dampng parameter λ k s a postve coeffcent and t has several effects. When the current soluton s far from the correct one, a large dampng parameter s chosen so that the procedure tends toward the slow-convergent steepest descent method. On the other hand, when the current soluton s close to the correct one, the dampng parameter decreases and the LMA behaves lke a Newton method. In ths work, we used a publc doman MINPACK [4] after slghtly modfyng the package to calculate the Jacoban matrx by usng a forward-dfference approxmaton. The second dentfcaton technque consdered s a partcle swarm optmzaton (PSO) [5,6]. The PSO s a populaton-based evolutonary algorthm whch s nspred by the socal behavor of brds flockng for food. The poston of a brd, also referred to as a partcle, represents the current soluton to the optmzaton problem. The PSO utlzes swarm ntellgence to fnd the best place n the search. Durng each epoch, all the partcles are accelerated toward ther own best poston and the global best t poston found so far by the swarm. Ths s acheved by calculatng a new velocty of each partcle ( v + ) accordng to three observatons: ts current velocty ( v t ), the dstance between each partcle s current t poston ( q ) and ts prevous best poston ( the swarm: t p ), and the dstance from the global best poston ( t t t t t t g t p ) n g v + = ωv + c r( p q ) + c r ( p q ) (9) where s a partcle ndex; ω s an nerta weght; c and c represent the weghtng factors that pull each partcle toward ts prevous best poston and the global best poston; r and r are random numbers unformly dstrbuted on the nterval [,]. The nerta weght plays an mportant role to balance the global and local search abltes; a large nerta weght facltates a global exploraton, whle a small one tends to facltate a local exploraton. A preferred weghtng functon, where the nertal weght s lnearly decreased as the teraton proceeds, s descrbed as: ω ω ω T max mn = ωmax t () where ω max s the ntal weght; ω mn s the fnal weght; T s the maxmum teraton number; t s the current teraton number. Once the new velocty of each partcle s determned by usng Equaton (9), the partcles update ther poston usng the followng equaton: q = q + v () t+ t t+ In ths way, the algorthm could converge toward a global soluton of the gven problem. The evoluton s contnued untl the ftness value reaches the preset value or the maxmum teratons are reached. Fgure 7 shows how the PSO-based parameter dentfcaton works and the evolutonary optmzaton steps of the PSO are gven below:

11 Sensors, 876 Step : Generate swarm and ntalze partcles n the swarm wth random postons and veloctes. Step : For each partcle, evaluate the ftness functon. Step 3: Memorze best solutons and a global best soluton n the swarm. Step 4: For each partcle, update poston and velocty by usng Equatons (9) and (). Step 5: Repeat Step untl predefned condtons are satsfed. Fgure 7. Schematc dagram of the PSO-based parameter dentfcaton. Intal Partcles... n q Measured Values... Reference Coordnates... Model parameters q q { θ, X, Y } f f P( x, y, z) Hand-mounted Laser-vson Sensor Model P ˆ ( x, y, z) Ftness Update Partcles (velocty, poston) Memorze Best Solutons Ftness Evaluaton 4. Smulatons In ths secton, we perform smulatons to compare the performance of the two optmzaton technques. Snce the exact soluton s known n the case of a smulaton experment, t s possble for us to compare the algorthms as to how the found solutons are close to the true optmum. At frst, the synthetc data was generated as follows: gven pre-specfed laser-vson sensor model parameters and a certan robot pose, we compute sets of data usng Equaton (9) n whch 3D coordnates of the laser ponts wth respect to the robot base frame correspond to randomly generated D mage coordnates n the pxel frame. The frst 5 sets of data are used to dentfy the model parameters whle the other 5 sets of data are used to evaluate the ftness of the found solutons. In order to nvestgate the nfluence of the ntal canddate solutons on the convergence performance, we used a control parameter, s whch determnes the sze of the ntal parameter bounds. Accordngly, the ntal solutons are randomly generated from wthn a certan parameter bound as: q = q [ + (rand().5) s] () where the functon, rand(), calculates a unformly dstrbuted random number n the range [,]; q s a th nomnal parameter used to generate the smulaton data. Snce some nomnal model parameters such as ω, φ, ϕ, and t y are assumed to be zero, we select ther bounds manually to avod a null range.

12 Sensors, 876 Next, we set the control parameters of the two optmzaton technques. In PSO, we used a swarm sze of 5, a maxmum nerta weght of.9, a mnmum nerta weght of.4, a maxmum velocty of., and a maxmum teraton of 5,. As for the LMA, the maxmum teratons are set as 3 tmes the number of model parameters. Snce the appled optmzaton technques set the ntal guesses of the parameters n a random manner, they may seek out dfferent mnma dependng on the ntal condtons. Therefore, we performed several runs for each algorthm to evaluate the performance:, runs for NLSO and runs for PSO. In addton to these algorthms, a random search (RS) was used only as a method to compare other algorthms. In the RS, an evaluaton starts wth parameters selected by usng Equaton () and when a better soluton s found, t replaces the current soluton. We performed,, evaluatons durng a run. The parameter dentfcaton results are lsted n Table. As for the NLSO, t s possble to fnd suffcently good solutons only when the ntal estmates are close to the exact solutons. As the ntal selecton spaces ncrease, however, the technque has a consderably ncreased tendency to get trapped n the local mnma or not to converge at all. Furthermore, even n the case wth a small range of the ntal parameters, the average root-mean-square error (rms) of NLSO s consderably hgh compared to that of the PSO. It ndcates that the optmzaton problem consdered has the characterstcs of a hghly nonconvex and multmodal landscape, even near the global optmum. On the contrary, the PSO consstently found a soluton close to a true optmum, although the best ftness value slghtly ncreased as the parameter bounds ncreased. The average rms of the PSO wth s =. (t means that the ntal selecton bounds are enlarged by up to ±% of nomnal parameters) s smlar to that of the NLSO wth s =.. Ths shows that the PSO can dentfy HMLVS parameters wth small errors wthout the need for good ntal estmates and that the enlargng spaces have only a margnal mpact on the estmaton accuracy. Table. Estmaton results wth 5 sets of smulaton data and wth a dfferent number of evaluatons:,, evaluatons for RS,, runs for NLSO, and runs for PSO. Optmzaton method RS NLSO PSO Root-meansquare s error..... E( q ) best.5e- 7.48E E( q ) best.e-5.7e-4 7.5E-4 5.8E-3.49E- E( q ) worst E3 7.4E.E3 F ( q ) mean.e E 6.E 6.E F ( q ) stdev 3.3E E 4.6E F ( q ) best.69e E E-4.4E-3.55E-3 F ( q ) worst.5e-4.7e-4.e-3.4e-.96e- F ( q ) mean 9.34E-5.89E E E-3.6E- F ( q ) stdev 9.E-5.58E-4 6.5E E-3.3E-3

13 Sensors, 8763 In order to valdate and examne the relablty of the obtaned model parameters, we calculate the sum of squared error of randomly generated 5 sets of data whch s not used n the optmzaton by usng the model parameters wth both best ftness and worst ftness of the PSO. As shown n Fgure 8, there s no sgnfcant dfference between two results wth dfferent data sets even though the sum of squared error slghtly ncreases as the parameter searchng bounds enlarges. Ths shows the fact that the selecton of nput data does not affect on the relablty of the estmaton accuracy. Fgure 8. Ftness evaluaton results of 5 sets of data not used n the parameter dentfcaton..3 Sum of squared error (mm )... Best ftness parameters Worst ftness parameters 5. Expermental Results Scale factor of parameter searchng space Fgure 9 llustrates the 5-DOF robot manpulator (SCORBOT ER-VII) and a reference object. A laser-vson sensor was nstalled at the last lnk of the robot manpulator. A laser reflecton mage s captured and dgtzed by a frame grabber (Meteor-II, Matrox) lnked to a monochrome CCD camera (XC-55, Sony). A checker board pattern wth a grd sze of 3 mm 3 mm s employed to provde reference postons. The left-top corner of the pattern s placed at the coordnate (7, 9, ) mm wth respect to the orgn of the robot base frame. A robot controller controls the poston and the velocty of the robot manpulator and t sends encoder readngs of each jont to the PC through the RS-3C lnk. As nput data for the optmzaton technques, we need three knds of nformaton: jont encoder readngs, real coordnates of the reference ponts, and ther correspondng mage coordnates. The reference pont s a corner pont of the checker board pattern, whch s gven as pror nformaton. The procedures for a data acquston are as follows: () Adjust the robot so that a horzontal lne of the laser strpe overlaps the corner ponts of the square pattern. () Capture an mage and then extract the pxel coordnate of the reference pont; record t n the memory of the PC. (3) Record all the jont angles of the manpulator sent from the jont controller. (4) Repeat () (3) untl we obtan the preset number of data sets.

14 Sensors, 8764 Fgure 9. Hand-mounted laser-vson sensor and the planar calbraton pattern. Laser strpe generator 5-DOF robot manpulator CCD camera Z B Y B X B reference pattern In order to effectvely extract a strpe of laser beam, we used a dfference mage, D(, xy ) whch subtracts one mage frame wthout a laser beam, F (, x y), from another mage frame wth a laser beam, F(, xy. ) Ths s acheved through togglng the on/off relay crcut used for provdng power to the laser strpe projector: F( x, y) F ( x, y) f F( x, y) > F ( x, y) Dxy (, ) = (3) f F( x, y) > F ( x, y) Calbraton ponts are obtaned through the matchng process between a strpe of laser beam and cross-ponts of contour lnes as shown n Fgure (d). The developed algorthms ncludng an mage processng, robot control procedure, and two optmzaton technques were mplemented by means of the C++ language. Fgure. Detecton result of the calbraton ponts. (a) mage (laser off) (b) mage (laser on) (c) dfference mage (d) calbraton ponts n the contour lne

15 Sensors, 8765 As descrbed n the prevous secton, the consdered parameter dentfcaton problem has hghly nonconvex and multmodal characterstcs such that the LMA often faled to fnd a good soluton, even startng at an ntal canddate soluton whch s near the true soluton. Besdes, the PSO consstently found a soluton close to a true optmum regardless of the searchng spaces. Therefore, we only consdered the PSO for a parameter dentfcaton n the followng experments. As control parameters of the PSO, we used a swarm sze of 5, a maxmum nerta weght of.9, a mnmum nerta weght.4, a maxmum velocty of., and a maxmum teraton of 5,. The nomnal values of the parameters are lsted n Table wth ther searchng bounds, whch are selected based on the specfcatons of the camera and the desgn parameters of the LVS. We determne the fnal parameters as those wth the lowest ftness value from dfferent runs. Table. Parameter dentfcaton results. Parameter Nomnal Parameter bounds of PSO values Lower Upper Best-ft parameters f S u S v C x C y k. 4.8E-4 H θ (rad) 5π/36 π/4.47 ω (rad) π/6 π/6.8e-3 φ (rad) π/6 π/6.5 φ (rad) π/6 π/6 6.8E-3 t x t y.3 t z Fgure shows the convergence performance of the objectve functon, where the sold lne shows the best ftness values from dfferent experments, and the dotted lne shows the worst ftness values. Fgure. Convergence of the ftness value. 6 5 best ftness worst ftness Ftness value Number of teraton

16 Sensors, 8766 Even though the ntal partcles (canddate solutons) are randomly generated from wthn the pre-defned range, these two curves are converged to a smlar ftness value after approxmately 3, teratons. Ths shows the fact that the PSO estmates the parameters wth a small error. It takes about 3 seconds to execute the 5, generatons. Fgure shows the average values and standard devaton of the dstance errors between the reference ponts and the calculated ponts of the expermental results. The accuracy s computed based on a root-mean-square error (rms) and ts average value of experments s.355 mm. By usng the constructed HMLVS wth parameters dentfed by the PSO, we measured the 3D range data of cylndrcal object wth holes, as shown n Fgure 3..8 Fgure. Calbraton results..6 Average Standard devaton Reference pont ndex Fgure 3. 3D range measurement of a cylndrcal object wth holes. To confrm the applcablty of the proposed scheme, we carred out experments measurng four corner ponts of the top surface of a gauge block whose sze s mm. A left-bottom of the block s placed at a pont (, 5, ) mm wth respect to the robot base frame. It s dfferent poston from the frst experment where the left-top corner of the pattern s placed at the coordnate (7, 9, ) mm. We start to move the robot from the robot home poston for each tral. We adjust the

17 Sensors, 8767 robot pose so as the laser strpe beam overlap the corner ponts of the top-sde of the object. The measurement results are lsted n Table 3, where x, y, z represent the Cartesan coordnate of the four corner ponts, and x, y, z are the measured mean range data. e and σ represent the mean value of the measurement error and standard devaton respectvely. Even though t s dfferent between the calbraton regon and the measurement regon, maxmum resdual error wth trals was about.7 mm. Ths shows the suggested algorthm s robust aganst the measurement locaton. Table 3. Measurement results for verfcaton. Ccorner ponts x y z x y z e σ left-top rght-top rght-bottom left-bottom Conclusons In ths paper we have proposed a new approach to the problem of a hand-mounted laser-vson sensor system calbraton based on a partcle swarm optmzaton. The laser-vson sensor, consstng of a camera wth a nonlnear radal lens dstorton and a laser strpe generator, was used as a sensor module of the robotc measurement system to measure the range data of an object n the robot base coordnate system; and t was modeled based on the forward knematcs and the optcal trangulaton prncple. The ntrnsc and extrnsc parameters of the hand-mounted laser-vson sensor were smultaneously dentfed through mnmzng the overall resdual errors between the known reference range data and the estmated data. Smulaton and expermental results show that the consdered parameter dentfcaton problem has hghly nonconvex and multmodal characterstcs, thus, the nonlnear least square optmzer often faled to fnd an optmal soluton, even when the ntal guesses of the model parameters were selected close to the true optmum. On the contrary, the proposed scheme based on the partcle swarm optmzer consstently found a stable soluton wthout any need for good ntal guesses of the model parameters; thus, t could be a promsng tool to dentfy the model parameters for a hand-mounted laser-vson sensor. Acknowledgements Ths work was supported by Nuclear Research & Development Program of Natonal Research Foundaton of Korea (NRF) funded by Mnstry of Educaton, Scence & Technology (MEST) Grant Code: -47. Ths work was also partally supported by the Korea government. MKE under the Grant No. I--- of the ETEP. A prelmnary verson of ths manuscrpt was presented at the Internatonal Conference on Convergence and Hybrd Informaton Technology (ICHIT ), 6 August n Daejeon, Korea and ths work represents an extenson of that paper n response to an honorable request by a conference commttee member for a journal publcaton after we obtaned some new results.

18 Sensors, 8768 References. Gong, C.; Yuan, J.; N, J. A Self-calbraton method for robotc measurement system. ASME J. Manufact. Sc. Eng. 4,, Char Y.Y.; Gweon, D.G. A calbraton and range-data extracton algorthm for an omndrectonal laser range fnder of a free-rangng moble robot. Mechatroncs 996, 6, Chen, C.H.; Kak, A.C. Modelng and calbraton of a structured lght scanner for 3-D robot vson. In Proceedngs of IEEE Internatonal Conference on Robotcs and Automaton, Ralegh, NC, USA, 3 March 3 Aprl 987; pp We, G.Q.; Hrznger, G. Actve self-calbraton of hand-mounted laser range fnders. IEEE Trans. Rob. Automat. 998, 4, Zhang, Q.; Pless, R. Extrnsc calbraton of a camera and laser range fnder (mproves camera calbraton). In Proceedngs of IEEE/RSJ Internatonal Conference on Intellgent Robotcs and Systems, Senda, Japan, 8 September October 4; pp Park, J.B.; Lee, S.H.; Lee, I.J. Precse 3D lug detecton sensor for automatc robot weldng usng a structured-lght vson system. Sensors 9, 9, Tsa, R.Y.; Lenz, R.K. A new technque for fully autonomous and effcent 3-D rbotcs hand/eye calbraton. IEEE J. Rob. Automat. 989, 5, Muñoz-Rodríguez, J.A. Calbraton modelng for moble vson based laser magng and approxmaton networks. J. Modern Opt., 57, Muñoz-Rodríguez, J.A. Moble calbraton based on laser metrology and approxmaton networks. Sensors,, Musa, E. Lne-laser-based yarn shadow sensng break sensor. Opt. Lasers Eng., 49, Salv, J.; Armangue, X.; Batlle, J. A comparatve revew of camera calbratng methods wth accuracy evaluaton. Patt. Recog., 35, Tsa, R.Y. A versatle camera calbraton technque for hgh-accuracy 3D vson metrology usng off-the-shelf TV cameras and lens. IEEE J. Rob. Automat. 987, 5, Nocedal, J.; Wrght, S.J. Numercal Optmzaton; Sprnger: New York, NY, USA, Moré, J.J.; Garbow, B.S.; Hllstrom, K.E. User Gude for MINPACK-; Report ANL-8-74p; Argonne Natonal Laboratory: Argonne, IL, USA, Kennedy, J.; Eberhart, R.C. Partcle swarm optmzaton. In Proceedngs of IEEE Internatonal Conference on Neural Networks, Perth, Australa, 7 November December 995; pp Sh, Y.; Eberhart, R.C. Emprcal study of partcle swarm optmzaton. In Proceedngs of the Congress on Evolutonary Computaton, Washngton, DC, USA, 6 9 July 999; pp by the authors; lcensee MDPI, Basel, Swtzerland. Ths artcle s an open access artcle dstrbuted under the terms and condtons of the Creatve Commons Attrbuton lcense (