DIRECT SENSOR-ORIENTED CALIBRATION OF THE PROJECTOR IN CODED STRUCTURED LIGHT SYSTEM

Transcription

1 DIRECT SENSOR-ORIENTED CALIBRATION OF THE PROJECTOR IN CODED STRUCTURED LIGHT SYSTEM M. Saadatseresht a, A. Jafar b a Center of Excellence for Surveyng Eng. and Dsaster Management, Unverstf Tehran, Iran, -msaadat@ut.ac.r b Azad Unversty, Khoy branch, -akbar_jafar@yahoo.com Commsson V, WG V/1 KEYWORDS: Metrology, Acquston, DEM/DTM, Pont Cloud, Sensor, Close Range, Industral, Stereoscopc ABSTRACT: Today, close range photogrammetry s known as an accurate and automatc 3D geometrc measurement method especally for 3D pont features (targets), but unfortunately t needs human supervson for 3D surface measurement due to nature of mage matchng problem. Therefore, n the last decade, coded structured lght technque surpasses as one of the man complementary automatc and precse 3D surface measurement methods n dfferent ndustral and medcal applcatons. Regardless of hardware consderatons, the accuracf structured lght method hghly depends on system calbraton ssue. Calbraton of the prector, whch s one of the major components of the structured lght system, s a very dffcult, tme-consumng and expensve process. In ths paper, the close range photogrammetry method s utlzed for drect calbraton of a prector. The experments demonstrate the proposed method not only s a low-cost and hgh-speed process but also s capable to calbrate the prector wth precson as 0 tmes of prector resoluton. Wth ths level of potental, we could acheve the relatve accuracf 1:13000 for 3D surface measurement. Keywords: Close range photogrammetry, structured lght, prector calbraton, geometrc dstorton. 1 INTRODUCTION Nowadays, 3D surface measurement s one of the mportant subjects n computer vson due to varetf ts applcatons n ndustry (such as qualty control and reverse engneerng), cultural hertage (such as 3D documentatons and vsualzatons) and medcne (such as body growng). Therefore, many nvestgatons n surface reconstructon and 3D nformaton extracton have been accomplshed under topc of "shape from X" that X s stereo mages, structured lght, texture, focus, shadng, reflecton and slhouette (Salv and et al, 004). Close range photogrammetry s one of the conventonal methods for ndrect 3D surface measurement but t s not generally an effcent method due to both reasons: (a) the fully automatc mage matchng has not been completely solved yet so t requres human supervson [Paar and et al, 001] (b) t s unavodable to put targets on the object surface especally n hgh precson applcatons. To prevent from these problems, t s possble to utlze the complementary methods such as structured lght method that s proper for measurement of a small and smple surface. To measure a large and/or complex surface, one of the best solutons could be fuson of close range photogrammetry and structured lght methods. The coded structured lght method measures a 3D surface based on magng of multple regular optcal pattern prected on the surface (Salv and et al, 004) (Fgure 1). The patterns encode and segment the surface to several small sub-regons as automatc correspondence problem for mage matchng s solved completely [Rocchn, 001]. It s noted that the prector s actually an nverse camera that prect a pattern nto a surface nstead of magng t from the surface. Therefore, an mage of a camera and a pattern of a prector are pseudo stereo mages that need nteror and exteror sensor orentaton or calbraton before mage ray correspondng and ntersecton for 3D measurement. Snce the calbraton of camera and prector has crtcal effects on 3D surface measurement accuracy; and camera calbraton s more known for users, ths paper only studes the prector calbraton ssue. Fgure 1. the system confguraton of the structured lght used n the study (pcture from [Tzung-Sz and et al, 000]) There are two methods for prector calbraton ncludng surface-related and sensor-related calbraton methods. In the surface-related prector calbraton, a known surface such as a flat plane s posed n dfferent known dstances and/or orentatons from prector. Then, the prector s calbrated based on prected pattern changes on the surface [Rocchn and et al, 001; Tzung-Sz and et al, 000; Snlapeecheewa and et al, 00]. Ths s entrely a laboratory method and requres a known surface and precse equpments of dstance and orentaton measurements whle ts computatons are low cost. The sensor-orented prector calbraton method could be mplemented wth both ndrect and drect technques. In ndrect mplementaton technque, a test feld ncludng a 119

2 The Internatonal Archves of the Photogrammetry, Remote Sensng and Spatal Informaton Scences. Vol. XXXVII. Part B5. Beng 008 regular 3D grd of ponts wth known coordnates s used for ndrect 3D coordnates measurement of prected patterns frst. Then they are prected up nto mage space and compared wth D coordnates of correspondng pattern's ponts to detect the geometrc dstorton of the prector [Alan and et al, 1995; Salvmas, 001; Guhrng and et al, 000]. Buldng of the test feld s expensve and ndrect measurng of 3D coordnates of prected pattern s naccurate due to nterpolaton error. In the drect mplementaton technque of the sensor-orented calbraton method, patterns are prected on unknown surfaces frst. Then the 3D prected patterns are drectly measured by close range photogrammetry (Zang and et al, 006). Ths method s a quck and precse soluton so we selected t for prector calbraton n our experments. As t s mentoned, t s possble to smply consder the prector as an nverse camera but t has some dfferences to camera so that t causes to be more dffcult ts calbraton. The frst and most mportant dffculty s that for camera calbraton, D mage coordnates are measured from known 3D object coordnates whlst t s vce versa for prector calbraton. It means we have to accurately measure the 3D prected pattern that s a tme-consumng and expensve task. In camera calbraton for dfferent camera placements, object ponts are statc to each others n the magng perod but they are dsplaced by prector movement. In addton, the stabltf lens system and bodf the prector s not as good as camera so that t has to be recalbrated n shorter perods. Also, nteror dstorton sze of the prector s larger than the camera so t needs more complex dstorton model wth more addtonal parameters. Further these facts, the prector calbraton s so dffcult because there s only a sngle pseudo mage (pattern) n our hand! In ths paper, n attenton to above dffcultes, we use our effort to calbrate a prector rapdly and accurately by a drect sensororented method. In other words, we utlze exstng close range photogrammetry facltes for prector calbraton. Next, the detals of drect sensor-orented prector calbraton are descrbed frst to make a background for reader to better understand our experments. DIRECT SENSOR-ORIENTED PROJECTOR CALIBRATION The basc steps of our proposed calbraton soluton are as follows: 1. At Frst, a D regular grd of smlar bnary crcular targets s drawn usng CAD software to be used as a pattern slde for prector. The sze of crcles should be at mnmum ten pxels on acqured mages and the grd cell sze should be more that two tmes of the crcle sze. It s noted that the grd must cover the whole area of the slde n order to be capable to accurately estmate the dstorton model parameters of the prector.. The pattern slde s prected on the center of a 3D wall corner wth three pars of rght planes. The reason to select a 3D corner as precton surface s to make 3D prected targets n dfferent depths. It causes to not only more magng network strength that leads to more accurate 3D measurement of the prected targets, but also the prector s optmally calbrated for 3D measurement of surfaces wthn predefned depth varatons. The best drecton of the prector optcal axs s 3D bsector of corner angle because the ellpse-shaped prected targets on rght plans have mnmal elongatons. It causes they could be measured wth hgher accuracy by close range photogrammetry method n the next step. 3. Several convergent mages are acqured from the prected targets by a sngle dgtal camera under some consderatons that s known n close range photogrammetry (Atknson, 1998): (1) The camera statons should make four to eght rays dstrbuted around each 3D prected targets. () In some camera statons, two or more mages should be acqured wth 90 degree of roll angles n order to reduce the dependency between nteror and exteror camera orentaton parameters. (3) The D mage ponts should cover whole frame of more mages to valdate the estmated dstorton parameters. (4) To acqure sem-bnary mages from the optcal targets, magng should be done under low lght of workspace, hgh lens aperture, and a fx zoom and focus. Besdes, the camera statons also are placed so that they don't obstacle the precton pyramd. 4. The mage coordnates of targets are measured for all mages. Then, the camera s calbrated through bundle adjustment and self-calbraton process whch s based on the geometrc mathematcal model between mage and object ponts coordnates. The outputs of network adjustment are estmaton value and covarance matrces of the camera calbraton parameters, exteror orentaton parameters of all mages, and 3D coordnates of the prected targets. For more nformaton about bundle adjustment and self-calbraton, please refer to basc books of close range photogrammetry such as (Atknson, 1998). 5. In the fnal step, the prector s drectly calbrated based on the known 3D coordnates of prected targets. As for prector calbraton, we used an exstng software of close range photogrammetry, named Australs (Fraser and et al, 000), and n the software t s not possble to work wth a sngle mage (slde of prector), both camera and prector are smultaneously ntroduced to the software, the former wth all mages and latter wth a sngle pseudo mage. The camera parameters and 3D targets coordnates are set to fx known varables. Then a second multple camera bundle adjustment and self-calbraton s accomplshed. The outputs are prector calbraton parameters and exteror orentaton of the prector and all camera mages. It s noted that, snce the prector has onlne mage n bundle adjustment, for more relabltf calbraton and more accuracf 3D coordnates of targets, the camera and prector calbraton has been done n two separate phases. Snce the prector s a reverse camera, n proposed soluton we consder ts calbraton model as conventonal model for dgtal camera n close range photogrammetry known as collnearty condton equatons (Atknson, 1998): c j[ rj,11( X X ) + rj,1 ( Y Y ) + rj,13 ( Z Z )] x x + Δx j = [ rj,31( X X ) + rj,3 ( Y Y ) + rj,33 ( Z Z )] c j[ rj,1( X X ) + rj, ( Y Y ) + rj,3( Z Z )] y y + Δy j = [ r ( X X ) + r ( Y Y ) + r ( Z Z )] j,31 n whch [X, Y, Z ] are coordnates of precton center of j th camera staton, r j ={r 3x3 } j s rotaton matrf j th camera staton, [X, Y, Z ] are object coordnates of th prected target, c j prncpal dstance of j th mage, [x, y ] are prncpal pont coordnates of j th mage, [x, y ] are mage coordnates of th prected target n j th mage, and [Δx, Δy ] are dstorton j,3 j,33 (1) 10

3 The Internatonal Archves of the Photogrammetry, Remote Sensng and Spatal Informaton Scences. Vol. XXXVII. Part B5. Beng 008 correctons for th prected target n j th mage. The geometrc dstorton correcton model s as follows (Atknson, 1998): Δx = ( x Δy = ( y n whch r x ) r = ( x 1 y ) r 1 ( K r 3 1 j ( K r 3 1 j + K + K 5 j 5 j x ) + ( y y ) r + K r ) + P [ r + ( x 7 3 j 7 3 j 1 j j r + K r ) + P [ r + ( y x ) ] + P j y ) ] + P ( x 1 j ( x x )( y x )( y y ) + B 1 j x + B y ) + B x j j y () n whch j, K j, K 3j are radal lens dstorton parameters, P 1j, P j are decenterng lens dstorton parameters, j, B j are mage affnty parameters n j th mage. In our proposed soluton, parameters of all mages are sensor nvarant. 3 EXPERIMENTS In our experments, we calbrated a DLP vdeo-prector from Infocus model X wth brghtness of 1500 lumens, resoluton of SVGA 800x600 by a consumer-grade dgtal camera from Canon model Powershot G3 wth 4 Mega Pxels mage qualty (7x1704 pxels wth 3μm CCD pxel sze), 4x zoom, and wdest focal length of 35mm (Fgure ). 3.1 Camera and Test-Feld Calbraton Before calbraton of the prector, the dstorton parameters of the camera and 3D coordnates of prected targets n test fled should be estmated through the frst bundle adjustment and self-calbraton process. The next step s second bundle adjustment and self-calbraton that s done for prector calbraton. To do so, a network of 3 convergent mages under consderatons mentoned n the prevous secton s acqured. Fgure 4 llustrates a general vew of the photogrammetrc network. The self-calbraton and free bundle adjustment of the network wth 6374 mage observatons, 568 unknown parameters, 11 constrants (5817 degree of freedom) was done by Australs software. Fgure. dgtal camera and vdeo prector used n our experments To calbrate the prector, t set out n three meters back dstance of a wall corner n sze of 1.5x1.5x1.5 cubc meters wth the axs along ts bsector (Fgure 3). Then, zoom and focus of the prector are set so that the targets sharply prect nto the whole corner space. Therefore, the proporton of object sze to object-prector dstance s approxmately 0.5 that s a rough proporton of pattern slde sze to prector prncpal dstance. Snce the vrtual sze of the pattern slde s measured on the montor about 5 mm, vrtual prncpal dstance of prector s approxmately 50 mm. Fgure 3. the pattern slde desgned for makng optcal targets (left) and the prected targets on the wall corner (rght) Fgure 4. sde vew (rght) and top vew (left) of the photogrammetrc network As the authors had calbrated the camera n ther last nvestgatons, we knew the camera stablty and ts sgnfcant calbraton parameters, so K, K 3, P 1, P parameters were removed from model then camera and test feld were calbrated. Table 1 shows the estmated value and precson of camera calbraton parameters, Table shows average precson of estmated coordnates of 130 prected targets n test feld, and Fgure 5 llustrates ther standard ellpsod errors. To calbrate the prector, these estmated values wll be ntroduced to the second bundle adjustment as known values. As t s seen from Fgure 5, the ellpsod errors are relatvely sphercal shape and smlar sze. Ths fact means the network s strong and results are relable. The desgned pattern s a grd of crcular targets wth dameters of 0.4 mm and dstance of mm on the montor wth vrtual sze of 5 mm. So t contans 13x10 = 130 optcal targets (Fgure 3). 11

4 The Internatonal Archves of the Photogrammetry, Remote Sensng and Spatal Informaton Scences. Vol. XXXVII. Part B5. Beng 008 Parameters c B Evaluated Value E E E-04 Standard Devaton 8.E-04(mm) 5.00E-04(mm) 5.14E-04(mm) 8.77E-06.15E-05.11E-05 Table 1. Evaluated calbraton parameters of the camera RMS X RMS Y RMS Z 0.04 RMS XYZ Table. The RMS of standard devatons of 3D target coordnates n the test fled (mm) Fgure 5. Standard ellpsod errors of targets n test fled (wth exaggeraton) Case A B C D RMS of Image Resduals (µm) Unt Weght Varance c K K 3 P 1 P B X σ x E X σ x E E E x σ x E E E E E-06.5E E E E x σ x E E E E E E E E E-1.7E E E E E-03.1E E-04 Table 3. The result of four prector self-calbraton processes 3. Prector Dstorton Model Determnaton As t s mentoned n secton, the general calbraton model of the prector s based on relaton. In ths model, there are 10 parameters. For the prector under calbraton, the statstcally nsgnfcant and dependent parameters should be removed form the model frst to derve the optmal calbraton model for that specfc prector. To do so, four bundle adjustment and self-calbraton processes were accomplshed wth predefned dfferent sgnfcant parameters: a) Onlne parameters for prector calbraton model (c). b) Three parameters for prector calbraton model (c,, ). c) Sx parameters for prector calbraton model (c,,,,, B ). d) All ten parameters for prector calbraton model (c,,,, K, K 3, P 1, P,, B ). In all computatons, all prevous known 3D prected targets are consdered as control ponts and camera calbraton parameters set to fx known varables. Then the prector are consdered as second sensor wth approxmate prncpal dstance of 50 mm, vrtual pxel sze of 30 μm and sensor sze of 800x600 pxels (vrtual sze of 4x18 mm ). As you can see n Fgure 6, the resdual vectors have a hgh level of systematc pattern n case A (magnfcaton 3). Therefore, a sngle parameter c for prector calbraton s not enough. In cases of B and C, also there s systematc error n resduals but wth less level (magnfcaton 51). In case D, the resduals are random vectors wthout detectable systematc error pattern. Therefore, the general mathematcal model wth ten parameters seems to be optmal for prector calbraton. The smlar result could be proved from Table 3 usng unt weght varance of adjustment. As t s known, f systematc error remans n observatons (or resduals), then unt weght varance of adjustment wll be rejected n related statstcal tests. Therefore, n Table 3, unt weght varance from case A to case D s sequentally reduced from 5 to unt. It proved there s not systematc error n observatons n case D. Our fnal result s that all ten parameters are sgnfcant. The column D of Table 3 shows the estmated values and ther standard devatons of calbraton parameters. 1

5 The Internatonal Archves of the Photogrammetry, Remote Sensng and Spatal Informaton Scences. Vol. XXXVII. Part B5. Beng 008 a:3x d:51x b:51x c:51x relatve accuracf 3D surface measurement s 0.3 mm for 3 m about 1: Also, the level of mage space error showd n Table 5 s derved n case that the vrtual pxel sze of prector s 30 μm. It means the mage relatve accuracf prector calbraton s μm for 30 μm about 0 tmes of prector resoluton. Fgure 6: the mage resduals n prectors for four dstorton models 3.3 Accuracy Evaluaton of the Proposed Calbraton To study the accuracf prector calbraton wth ten mentoned parameters, we dvded 130 targets nto two homogeneous and complementary groups: 75 control ponts and 75 check ponts. The control ponts were used for calbraton of the prector whle the check ponts were used for accuracy evaluaton of the calbraton result. Table 4 shows the calbraton parameters estmated by the control ponts. Parameters C K K 3 P 1 P B Evaluated value E E E-11.45E E E E-04 Standard devaton 4.069e-00 (mm) 4.394e-00 (mm) 5.537e-00 (mm) 1.67E E E E E E E-04 Table 4: the value of calbraton parameters estmated by 75 control ponts Fgure 7: (down) resdual vectors of check ponts (blue) and control ponts (red) n mage space and (up) object space after blunder removal (mag. factor: 1000) Object Space (mm) Image Space (µm) d X d Y d Z d XYZ d x d y d xy Control Ponts Mean RMS Check Ponts wth blunders wthout blunders Mean RMS Mean RMS Table 5: accuracf mage and object coordnates on control and check ponts Then the parameters of the calbrated prector was fxed and camera/prector bundle adjustment was repeated wth all control and check pont observatons. Table 5 shows the result of coordnates comparson n mage and object space for both control and check ponts. It s noted that the level of object space errors showed n Table 5 s derved n case that prectorobject dstance s three meters. In other words, the object 13 Parameters c K K 3 P 1 P B Evaluated value E E E E E E E-04 Standard devaton.61e-0(mm) 3.08E-0(mm) 3.15E-0(mm) 1.09E-06.96E E E E E E-05 Table 6: the fnal estmated value of prector calbraton parameters wth all targets Fgure 7 llustrates the resdual vectors of control/check ponts n mage/object spaces after blunder removal. As t can be seen, some blunder vectors have been detected and removed. The blunders are unavodable for optcal targets because the surface of precton for some targets may be have small geometrc

6 The Internatonal Archves of the Photogrammetry, Remote Sensng and Spatal Informaton Scences. Vol. XXXVII. Part B5. Beng 008 rregulartes that make local errors for these targets. For more precse calbraton of prector and 3D surface measurements, these prected targets have been detected and elmnated from observatons. To evaluate the fnal value of calbraton parameters, all 130 optcal targets are consdered as control ponts and partcpate n bundle adjustment and prector self-calbraton. To have more precse and robust estmaton, the large mage resduals automatcally detected and elmnated n Australs software. Table 6 shows the fnal precton calbraton result that has small dfferences to Table 3-column D and Table 4. 4 CONCLUSION AND FUTURE WORKS One of the most mportant ponts to acheve a hgh accuracy for 3D surface measurement usng a structured lght system s precse calbraton of the system. It ncludes nteror calbraton of camera and prector and ther relatve calbraton. In ths regard, nteror calbraton of prector s more dffcult that camera and takes more tme and cost. In ths paper, the close range photogrammetry method s used to calbrate the prector rapdly and accurately. At frst, selecton of a proper calbraton model wth ten parameters to model dstortons of prector was studed through four experments. Then, the prector was calbrated wth mentoned dstorton model rapdly. The tme of magng (33 mages) and computatons (about 6400 observatons and 600 unknowns) wth Australs software was n range of ten mnutes. The result of our experments demonstrates prector can be calbrated n accuracf 0 tmes of ts resoluton by close range photogrammetry. Ths calbrated prector also can measure 3D surfaces n accuracf 1:13000 of prector-object dstance. In our proposed calbraton process, we expected f a more geometrcally robust prector wth hgher resoluton and depth of focus s utlzed and f the prector s nstalled n a fxed poston, t may be to get more precse results under hgher measurement speed. Therefore, our future research s to buld a structured lght system wth hgher qualtf hardware and especal automatc processng software. The fuson of close range photogrammetry and structure lght methods s another topc for our future research to be able to measure 3D surface of large and/or complebjects. REFERENCES Rocchn, C., Cgnon, P., Montan, C., Png, P., 001. A low cost 3D scanner based on structured lght. Eurographcs, Vol. 0, No. 3. Tzung-Sz, S., Cha-Hsang, M., 000. Dgtal prector calbraton for 3D actve vson systems. Transacton of the ASME, Vol.14: Mclover, A.M., Valkenburg, R.,J., Calbratng a structured lght system. Image & Vson Computng New Zealand, pp Paar, G., Rottenstener, F., Pölzletner, W., 001. Image Matchng Strateges. In: Kropatsch W.G. and Bschof H. (Eds.), Dgtal Image Analyss. Selected Technques and Applcatons. Sprnger-Verlag New York, ISBN pp Snlapeecheewa, C., Takamasu, K., 00. 3D profle measurement by color pattern precton and system calbraton. IEEE ICIT 0, Bangkok, THAILAND Salvmas, J., 001. An approach to coded structured lght to obtan three dmensonal nformaton. PhD Thess, Unverstf Grona. Salv, J., Pag'es, J., Batlle, J., 004. Pattern codfcaton strateges n structured lght systems. Pattern Recognton, 37(4): Guhrng, J., Bremmer, C., Bohm, J., Frtsch, D., 000. Data processng and calbraton of cross-pattern strpe prector. IAPRS, Amsterdam, 33(5). Zang, S., Huang, P.S., 006. Novel method for structured lght system calbraton. Opt. Eng., Vol. 45, No. 8. Fraser, C., Edmundson, K., 000. Desgn and mplementaton of a computatonal processng system for off-lne dgtal closerange photogrammetry. ISPRS Journal of Photogrammetry and Remote Sensng, 55: ACKNOWLEDGEMENTS Fnancal support of the research councl of the Unverstf Tehran under a research prect, number /1/0, s kndly apprecated. Atknson, K.B Close range photogrammetry and machne vson. Whttles Publshng, Scotland, 384 pp. 14