Projector Calibration for 3D Scanning Using Virtual Target Images

Transcription

1 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1, JANUARY 2012 / 125 DOI: /s rojetor Calibration for 3D Sanning Using Virtual Target Images afeez Anar 1, Irfanud Din 1 and Kang ark 2,# 1 Graduate Shool of Mehanial Engineering, Myongji University, San 38-2, Namdong, Cheoin-gu, Yongin, South Korea, Deartment of Mehanial Engineering, Myongji University, San 38-2, Namdong, Cheoin-gu, Yongin, South Korea, # Corresonding Author / kang@mju.a.kr, TEL: , FAX: KEYWORDS: rojetor alibration, Intrinsi arameters, Etrinsi arameters, omograhy In this ork e roose a novel method to alibrate the rojetor. Calibration of the rojetor deals ith the alulation of geometri arameters of the rojetor hih are intrinsi and etrinsi arameters. An un-alibrated amera assists the roess of finding the geometri arameters for rojetor. This method eloits the fat that a rojetor an be treated as inverse amera. The amera atures the sene and saves it as an image hile the rojetor rojets an image on the sene. Camera to rojetor transformation is used to make the rojetor able to see the alibration attern. Then the rojetor is alibrated in a similar fashion like amera. Real data is used to evaluate the roosed method of rojetor alibration and good results are obtained. Manusrit reeived: Deember 30, 2010 / Aeted: August 2, Introdution 3D shae measurement and reonstrution or simly 3D sanning, has beome one of the hottest fields in Comuter Vision and Robotis during ast fe years. Researhers from various fields like Comuter Vision, Robotis, Mehatronis, Intelligent Manufaturing Systems, and Alied Otis have orked enormously to find more robust, less omle, and fast 3D sanning tehniques. These tehniques are being adated by medial, raid rototying, defense and other numerous industries. In Manufaturing here automati insetion is used these days, 3D sanning an erform ell. 1 On the basis of their harateristis, these tehniques are divided into to subgrous; assive sanning and ative sanning. assive sanning does not require diret ontrol of illumination soure, instead relies entirely on ambient light. 2 This system usually makes use of to or more ameras to measure and reover the 3D geometry. The image of the objet is atured by both the ameras from different ositions and orientations simultaneously. Triangulation is then used for measurement of the 3D geometry. The bottle nek in the stereovision system is the orresondene. This is to find the orresonding oints in the rojetion of the sene in one amera to the oints in the other amera. To oe ith the orresondene roblem, various image roessing tehniques are used. The orresondene roblem is not involved in ative sanning tehniques. In this method the rojetor rojets a strutured light on a 3D geometry hih is atured by a single amera. During the ast fe years a lot of ork has been done on this tehnique and many eole have ome u ith some very diverse ideas. This tehnique of reonstruting and measuring the 3D geometry is fast, robust, and ineensive. Eseially these days the dereasing ries of rojetors and CCD ameras have made it easy to have a 3D measurement system. But before doing any re-onstrution and measurement roess the rojetor and the amera system must be alibrated. The amera alibration has been heavily studied by the researhers therefore a vast variety of algorithms eist for it. This researh fouses on the geometri alibration of the rojetor hih deals ith the alulation of the intrinsi and etrinsi arameters of the rojetor. Many researhers have orked on the geometri alibration of the rojetor. Zhang and uang 3 have given the idea to ature images ith a rojetor. The rojetor is used to ature images like a amera, through this ay the rojetor an be alibrated like a amera. They have made the rojetor see the hessboard attern ith hel of a amera by using the sinusoidal attern rojetion tehnique. The main diffiulty lies in making the seial setu of hite and red light illumination. Li and Shi 4 also roosed the alulation of the DMD image i.e. the image atured by the rojetor using vertial and horizontal fringe atterns, hih makes it a time onsuming method. Gao and Wang 5 have done the rojetor alibration using omograhies. The roblem involved in this idea is the use of red and blue attern like Zhang and uang. 3 They also use too many KSE and Sringer 2012

2 126 / JANUARY 2012 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 images of the rinted hessboard beause hen they move the sreen to a different osition they must take the image of the rinted hessboard that is attahed to the sreen in order to alulate the etrinsi matri. Liao and Cai 6 have roosed the rojetor alibration by unouling the rojetor and the amera. The results they had are good but they use a alibrated amera as ell. The other limitation is that they use the image of the rinted hessboard for eah osition of the lane. In this aer the rojetor alibration for the 3D measurement system is done based on the riniles of the amera alibration. The roosed tehnique uses an un-alibrated amera for rojetor alibration. The amera and the sreen are both fied hile the rojetor is moved to different ositions and orientations in order to generate the virtual images. Virtual images are those images hih are seen by the rojetor. This seems a bit imossible but in fat these images an be etrated ith the hel of the amera. The roosed tehnique uses an un-alibrated amera for rojetor alibration thus avoiding the errors of amera alibration to aumulate in the alulated arameters of the rojetor. Seondly, in this method the sreen and the amera are both stationary due to hih the transformation beteen the amera and the sreen is onstant. The etrinsi arameters of the amera are not alulated again and again as is done in 5,6 hen the sreen is moved to different ositions and orientations. 2. Bakground Knoledge 2.1 Camera Model Imliation to rojetor Fig. 1 shos the ersetive rojetion models of the amera and the rojetor along ith their oordinate systems. Let [X Y Z 1] T be a 3D oint in orld oordinate system and [u v 1] T be the 2D oint in the amera image lane. is rojeted on u to a sale fator s aording to the inhole amera model, as follos; sm [ R t ] (1) here M is the set of intrinsi arameters of the amera and is given as oint in the rojetor image lane oint in the orld oordinate system M f 0 0 f y y Z Sreen Fig. 1 Mathematial Model of the Camera and the rojetor Y X Z am Y X orld roj Z Y X In intrinsi matri (, y ) are the o-ordinates of the rinial fous and (f, f y ) are the foal lengths along the and y aes of the image lane resetively. [R t ] reresents the transformation alled etrinsi arameters. It gives the rotation and translation beteen the orld and the amera oordinate systems. The rojetor an be onsidered as a amera that ats in reverse manner. It rojets the image instead of aturing it. Therefore the inhole amera model an be alied to the rojetor as ell. Let [u v 1] T be a 2D oint in rojetor s image lane hih is rojeted to a 3D oint [X Y Z 1] T in orld lane. For this 2D-3D oint air Eq. (1) an be modified as sm [ R t ] (2) ere M is the set of intrinsi arameters and [R t ] is the set of etrinsi arameters of the rojetor. 2.2 Zhang s method A lot of ork has been done on amera alibration during the ast fe deades. The latest tehnique that is used by most of the researhers is Zhang s method. 7 This tehnique uses inhole amera model hih has foal length, iel size, and skes fator as intrinsi arameters and the translation and rotation of the amera referene frame ith reset to the orld referene frame as etrinsi arameters. The alibration is simly a roess that finds the intrinsi and etrinsi arameters of the amera. This method assumes the amera ith no distortions hile solving for alibration arameters. Based on Zhang s method the alibration roess is rogrammed using the folloing stes. 1. A regular shaed objet like a hessboard is attahed to a flat and smooth lank. 2. Images of the objet are atured at different ositions and orientations. 3. The feature oints in the image are deteted using a funtion of OenCV and stored as image oints. 4. The oordinates of the feature oints in orld oordinate system are stored as objet oints. 5. Both of the matries are rovided to the main alibration funtion in OenCV to find the intrinsi and distortion arameters of the amera. 6. The set of etrinsi arameters of the amera are then determined ith the hel of the intrinsi arameters. 2.3 lanar omograhy omograhy is a nonsingular 3 3 matri, hih defines a homogeneous linear transformation from a lane to another in a rojetive sae. 10 Let the y lane of the orld oordinate system oinide ith the sreen s lane as shon in Fig. 1, then a 3D oint on the sreen beomes [X Y Z 1] T and aording to inhole amera model X u r r r t Y v sm r r r t y 0 1 r r r t z 1

3 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 JANUARY 2012 / 127 or u r r t X v sm r r t Y y 1 r r t z Let be the omograhy from orld lane to the amera image lane hih is given as h h h r r t y h h h r r t z h h h sm r r t A oint is rojeted to, a oint on the orld lane (i.e. sreen). And this oint is re-rojeted to another oint in the amera image lane as shon in Fig. 1. Then and are related to eah other aording to the folloing relation. (3) As it is stated before that the rojetor an be onsidered as a amera ating in reverse order so the inhole amera model an also be alied to it. Therefore a oint on rojetor s image lane an be related to a oint on orld lane and this relationshi an be resented in the folloing form (4) here is the lanar omograhy from the orld lane to rojetor image lane. No ombining Eq. (3) and Eq. (4), e get ( ) 1 or (5) Let be the omograhy from the amera to the rojetor hih is given as No utting Eq. (6) in Eq. (5) e get (6) (7) As the omograhy beteen the amera image lane and the rojetor image lane is a 3 3 matri ith eight unknons therefore at least four airs of orresonding oints in both the lanes are suffiient to find it. omograhy is alulated using the funtion in OenCV hih hooses normalization here ninth arameter of the omograhy matri is Virtual alibration attern generation virtual alibration atterns. These are the images of the rinted hessboard seen by the rojetor. In this setion the hole roedure for generating the virtual alibration atterns ill be disussed. The method roosed in this researh uses the rotation and translation of the rojetor instead of the sreen. The sreen and the amera both are fied. The amera is tilted and fied at 20º ith reset to the stationary sreen. First of all the rinted hessboard is attahed to the sreen and its image is atured ith the amera. The rinted hessboard is then removed from the sreen. The rojetor is roughly brought to a fronto-arallel osition.r.t the sreen. A knon hessboard attern is then rojeted by the rojetor on the sreen and atured ith the amera. Let be the knon attern that ill be rojeted by the rojeted rojetor on the sreen and be the image of the rojeted rojeted attern atured by the amera as shon in Fig. 2. Then Eq. (7) an be ritten as rojeted rojeted The orners of both the hessboards are then deteted using the funtions in OenCV. These orners are then used to determine the omograhy beteen the fronto-arallel osition of the rojetor and the amera. Let the image of the rinted hessboard attern atured by the amera be and the virtual alibration attern be, then rnted rnted Eq. (7) an be ritten as rnted rnted Sine the omograh is knon, it an be alied to the image of the rinted hessboard atured by the amera to generate the virtual alibration attern. The images of the rinted hessboard are atured by the amera and the orresonding virtual alibration attern is shon in Fig. 3. Sreen rojeted rojeted rojeted Sreen Camera rojetor Camera rojetor For rojeted attern For rinted attern Fig. 2 rojeted and rinted hessboard oints rntd rntd rnted The images of the hessboard ith different ositions and orientations an be used to alibrate the amera. As it is mentioned earlier that the rojetor an be onsidered as a reverse amera, so if the rojetor is made able to ature the images of the rinted hessboard attern then it an also be alibrated like a amera using Zhang s method. It sounds a bit imossible but in fat it an be done ith the hel of a amera hih ould ork along ith the rojetor to generate these virtual images hih are named as rnted * rnted Fig. 3 Image of the rinted hessboard seen by the amera and virtual alibration attern for fronto-arallel ose of the rojetor

4 128 / JANUARY 2012 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 In Fig. 4, let the fronto-arallel ose of the rojetor be named as {1}, then for this ose Eq. (6) an be ritten as 1 1 or 1 1 No the rojetor is moved to another ose. Let this ose of rojetor be named as {2}, then for this osition Eq. (6) an be ritten as or 2 (8) (9) Sine L..S of Eq. (8) and Eq. (9) are the same hih means that amera to sreen omograhy is onstant, so ombining both of them gives, Sreen { } rinted hessboard (10) No let the virtual alibration attern at fronto-arallel osition {1} 1 be, and the virtual alibration attern at ne ose {2} be rnted 2 then Eq. (7) an be ritten as rnted rntd 1 rntd The omograhy beteen to oses of the rojetor an be alulated using Eq. (10) and the virtual alibration attern for the fronto-arallel ose is already alulated hih ill allo alulating the virtual alibration attern for the seond ose easily. To rite Eq. (10) more generally, (11) i i 1 1 From Eq. (11) it an be summarized that the omograhy beteen the first ose and any arbitrary ose of the rojetor an be alulated using the rojetor to amera omograhies of both the oses. Then using this omograhy the virtual alibration attern for any arbitrary ose an be alulated from the virtual alibration attern for the fronto-arallel ose. In Fig. 5, some virtual alibration attern for some arbitrary rojetor oses are shon hih are generated on the omograhy alulated by Eq. (11) and the virtual alibration attern for the first ose. start STE_1: Bring rojetor to fronto-arallel osition.r.t the sreen {1} Camera {} 1 ro 1 _1 ro _ 2 ro_1 Zro i i Z2 Z Z Z1 Z i rojetor rojetor rojetor X ose 2 ose 1 { i } X i ose i ro_ 2 X { 1} X1 { 2} 2 X2 1 i ro_1 X ro _ 2 ro _ 1 ro i Y2 Y Y1 Y Y i ro_ 2 ro i ro_1 _ ro_ 1 2 _2 ro _i _ 2 i ro i i 1 1 ro _1 _1 1 ro _ 1 STE_2: Take image of the rinted hessboard attahed to the sreen. { } rnted STE_3: rojet a knon hessboard attern from rojetor and ature ith amera. 1 { rojeted } STE_4: Calulate amera to rojetor omograhy using image set from ste 3. { } 1 Fig. 4 rojetor oses and omograhies from amera to rojetor ose 1, from rojetor ose 1 to ose 2 and ose i STE_5: Transform the image taken in ste 2 to rojetor image lane using omograhy from ste 4. rnted 1 { } STE_6: Move rojetor to another osition i. {i} 2 rnted 3 rnted ro _2 2 _1 1 ro _3 _1 ro 3 1 ro _4 _1 ro 4 1 ro * * * 1 rnted 1 rnted ii+1 STE_7: rojet a knon attern and ature ith amera and alulate amera to rojetor omograhy. STE_8: Calulate omograhy from first rojetor osition to ith rojetor osition using amera to rojetor omograhies of both ositions. STE_9: Calulate the ne image of the rinted hessboard for ne osition using omograhy from revious ste and image from ste 5. YES i { } i { } 1 i<n NO 4 rnted 1 rnted STE_10: Use all the virtually generated images for rojetor alibration using Zhang s method. Fig. 5 Resultant virtual alibration atterns of different oses after alying omograhies on virtual alibration attern of ose 1 END Fig. 6 Flo hart for the generation of virtual alibration atterns

5 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 JANUARY 2012 / 129 The flo hart for the generation of the virtual alibration atterns is shon in Fig. 6. The hole roess an be summarized in the folloing stes. 1. Attah a rinted hessboard to the stationary sreen and take an image of the rinted hessboard ith a amera hih is tilted at some angle.r.t the sreen. 2. Remove the rinted hessboard from the sreen. Bring the rojetor roughly to fronto-arallel osition.r.t the sreen. 3. rojet a knon attern on the sreen and take an image ith the amera. 4. Calulate the omograhy beteen the amera and the rojetor using the air of images from ste Aly omograhy from ste 4 to the image of rinted hessboard from ste 1 to alulate the virtual alibration attern for the fronto-arallel ose. 6. Move the rojetor to another ose. 7. rojet the knon attern and ature ith the amera. 8. Calulate omograhy beteen the ne ose of the rojetor and amera using image air from ste Calulate inter-ose omograhy for rojetor using omograhies from ste 4 and ste Aly omograhy from ste 9 to virtual alibration attern of fronto-arallel osition to get virtual alibration attern for urrent ose. 11. Reeat from ste 6 to ste 10 until suffiient images are obtained for alibration. Etrinsi arameters: Re-rojetion Error Analysis The roosed method is based on the linear model of the rojetor. All of the alulations are based on the inhole amera model by negleting the lens distortion. The re-rojetion error for rojetor is also alulated. It is defined as the differene of the orners of the virtual alibration attern and those of the bak rojeted image of the rinted hessboard based on intrinsi and etrinsi arameters of the rojetor. The rojetor is brought to an arbitrary osition. For that seifi osition the virtual alibration attern is generated and then using this virtual alibration attern etrinsi arameters of the rojetor are alulated. The orners of this virtual alibration attern are deteted and stored as measured orners. The orners of the rinted hessboard hih are in millimeters are transformed 4. rojetor Calibration Eeriments and Results The virtual alibration atterns for all the oses of the rojetor are obtained using the omograhies and the virtual alibration attern from ose 1. All of these images are no used to alibrate the rojetor. For alibration the funtion in OenCV is used hih is based on Zhang s method. This funtion gives the intrinsi and distortion arameters as outut. In this ork the IDS ueye CCD amera is used. The resolution of this amera is The LCD rojetor used is ESON EB-1735W ith a resolution of As it an be seen that the hessboard used is of dimension 6 7 ith eah square of size 17mm 17mm. The setu is shon in Fig. 7. The orld o-ordinate system is attahed to the rinted hessboard attern. The uer left most orner of the hessboard attern is taken to be the origin. y aes are on the lane and z ais is erendiular to it. The o-ordinate systems of orld, amera, and rojetor are being shon in Fig. 1. The set of intrinsi and etrinsi arameters (for fronto-arallel ose only) of rojetor are given. rojetor intrinsi and etrinsi arameters Intrinsi arameters: Fig. 7 Camera rojetor setu, ith amera tilted at 20 and rojetor in fronto-arallel osition.r.t sreen y Re-rojetion error (iels) image0 image1 image2 image Fig. 8 Re-rojetion Error for the rojetor alibration using si virtual alibration atterns

6 130 / JANUARY 2012 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 to the image lane of the rojetor using intrinsi and etrinsi arameters of the rojetor and are named as alulated orners. The differene beteen these to tyes of orners is then alulated aording to Eq. (12), hih is knon as re-rojetion error. Let q al (m al,n al ) be the alulated image oint and q meas (m meas,n meas ) be the measured image oint. Then the rerojetion error is alulated using the folloing equation. m m m al meas (12) n n n al The re-rojetion error for rojetor alibration using the virtual alibration atterns is resented in Fig. 8. It is alulated for four different images ith eah image having a different symbol in the grah. The statistis shon in Table 1 are generated from the rerojetion error grah of Fig. 8. The standard deviations for rerojetion error for the rojetor in and y diretion are and resetively. For omarison the re-rojetion error for amera alibration is also shon in Fig. 9. The standard deviations of re-rojetion error for the amera in and y diretion are and resetively. The result of the rojetor alibration in Fig. 8 shos a little bit greater error distribution than that of amera alibration in Fig. 9 sine the amera as used to get data for the rojetor alibration. oever, the result of the rojetor alibration is still aurate enough to be used in 3D sanning system. In omarison ith revious researh, 5 our alibration method using virtual target images rodues the similar auray; hoever our method is muh easier to use sine the amera and the sreen are fied and the Table 1 Re-rojetion statistis of si virtual alibration atterns Image no Mean () STD () Mean (y) STD (y) iels iels iels iels Image Image Image Image y meas Rerojetion error (in iel) Fig. 9 Re-rojetion Error for an ordinary amera image 0 image 1 image 2 image 3 image 4 amera takes only one image of the rinted alibration target attahed to the sreen. This is due to the fat that instead of sreen, the rojetor is moved to different ositions and orientations. 5. Conlusion In 3D sanning system rojetor alibration is an imortant ste. In this ork an innovative method for the rojetor alibration is roosed. The amera to rojetor omograhy is used to generate images seen by the rojetor. The amera and the sreen both remain stationary during the roess of generating images. Instead the rojetor is moved to different osition and orientation hih has the advantage to avoid moving the big sreen. The geometri arameters of the amera are neither alulated nor used. The generated images are then used to determine the geometri arameters of the rojetor. Re-rojetion error is alulated and omared ith the re-rojetion error of a amera. ACKNOWLEDGEMENT This ork as suorted in art by the Korea Ministry of Knoledge Eonomy, under Grant of the Strategi Tehnology Develoment rojet on Biomedial Sulier (Develoment of the Digital Fusion Artifiial Tooth Treatment Suorting System). REFERENCES 1. Song, J. Y., ark,. Y., Kim,. J. and Jung, Y. W., Develoment of Defet Insetion System for D ITO atterned Glass, Int. J. reis. Eng. Manuf., Vol. 7, No. 3, , Lanman, D. and Taubin, G., Build Your On 3D Sanner: 3D hotograhy for Beginners, SIGGRA,. 1-94, Zhang, S. and uang,. S., Novel method for strutured light system alibration, Ot. Eng., Vol. 45, No. 8, aer No , Li, Z., Shi, Y., Wang, C. and Wang, Y., Aurate alibration method for a strutured light system, Ot. Eng., Vol. 47, No. 5, aer No , Gao, W., Wang, L. and u, Z., Fleible method for strutured light system alibration, Ot. Eng., Vol. 47, No. 8, aer No , Liao, J. and Cai, L., A Calibration Method for Unouling rojetor and Camera of a Strutured Light System, ro. of IEEE/ASME International Conferene on Advaned Intelligent Mehatronis, , Zhang, Z., Fleible amera alibration by vieing a lane from unknon orientations, ro. of the 7 th IEEE International Conferene on Comuter Vision, Vol. 1, , 1999.

7 INTERNATIONAL JOURNAL OF RECISION ENGINEERING AND MANUFACTURING Vol. 13, No. 1 JANUARY 2012 / Bradski, G. and Kaehler, A., Learning OenCV: Comuter Vision ith the OenCV Library, O Reilly Media, Truo, E. and Verri, A., Introdutory tehniques for 3-D omuter vision, rentie all, artley, R. and Zisserman, A., Multile Vie Geometry in Comuter Vision, 2nd ed., Cambridge University ress, Drareni, J., Roy, S. and Sturm,., Geometri Video rojetor Auto-Calibration, Comuter Vision and attern Reognition Workshos, , 2009.