Accuracy Improvement in Camera Calibration

Transcription

1 Accuracy Improvemet i Camera Calibratio FaJie L Qi Zag ad Reihard Klette CITR, Computer Sciece Departmet The Uiversity of Aucklad Tamaki Campus, Aucklad, New Zealad fli006, qza001@ec.aucklad.ac.z r.klette@aucklad.ac.z Abstract Camera calibratio is a ecessary ad critical step i 3D object aalysis. The accuracy of calibratio results will affect the object s positio i world coordiates, especially for 3D object trackig. I this paper, we preset a ew camera calibratio approach, ad discuss its accuracy. We use 3D marks istead of 2D marks for calibratio. Our experimetal results show that our approach has the potetial to improve the calibratio accuracy. Keywords: 3D recostructio, camera calibratio. 1 Itroductio I the cotext of three-dimesioal machie visio, camera calibratio is a process for determiig the iteral geometric ad optical camera characteristics (itrisic parameters), ad the 3D positio ad orietatio data of the camera frame relative to a defied world coordiate system (extrisic parameters). A calibratio techique is based o kow 3D coordiates of geometrically cofigured poits. Here the 3D coordiates are usually refereced to the world coordiate system. The cofigured poits are commoly referred to as calibratio poits which are physically realized by calibratio marks o a calibratio object. Commo calibratio methods are DLT (Direct Liear Trasform) ad Tsai s method [1, 2]. The latter oe also models les distortio coefficiets ad the mappig of sesor elemets to a image buffer matrix; it requires at least seve accurately detected calibratio poits i a arbitrary but kow geometric cofiguratio. Usig Tsai s calibratio method, we ca trasfer 3D world coordiates ito image coordiates. But eve usig the same calibratio method, differeces i image acquisitio eviromets may affect the accuracy, such as distaces betwee camera ad object, the size ad umber of calibratio marks, or the size of calibratio objects. We evaluated Tsai s method with respect to such variatios, where 3D marks are used istead of the commo 2D calibratio marks. Sectio 2 reports about improvemets i calibratio accuracy; Sectio 3 specifies this ew alteratio of Tsai s method; Sectio 4 presets further experimetal results, ad Sectio 5 cotais our coclusios. 2 Accuracy Improvemet Calibratio accuracy will affect calculatios of object s positio ad trackig parameters. We summarize four methods for evaluatig camera calibratio accuracy: 1. Image Coordiates Error Statistics: This measuremet method aalyzes the distorted statistics o the image plae. Steps are: covert world coordiates ito camera coordiates, the ito udistorted sesor plae coordiates, the ito distorted sesor plae coordiates, the ito image coordiates. After all these coversios, determie s betwee ideal image coordiates ad actual locatios of data poits. 2. Udistorted Image Plae Error Statistics: This measuremet method aalyzes the udistorted statistics o the image plae. Steps are: covert world coordiates ito camera coordiates, the ito udistorted sesor plae coordiates; covert from 2D image coordiates ito distorted sesor plae coordiates, the ito udistorted sesor plae coordiates. After covertig, determie the betwee ideal ad actual locatio of the data poit. 3. Camera Coordiates Error Statistics: This measuremet method aalyzes the statistics i object space. Steps are: covert world coordiates ito camera coordiates; covert from 2D image coordiates ito distorted sesor plae coordiates, the ito udistorted sesor plae coordiates, the ito 3D camera coordiates. After covertig, the is defied by distaces of closest poits to ideal projectio rays i 3D camera coordiates space. 338 Image ad Visio Computig NZ

2 Method 3 is also kow as Normalized Calibratio Error [5]. We explored four differet schemes for improvig calibratio accuracy. Methods 3 ad 4 (i the order below) proved to be efficiet ad of practical use. 2.1 Subset Search It is commo to use all the calibratio marks to produce the calibratio data file. I practice, some of the image coordiates of projected calibratio marks deviate from their true values. Usig differet umbers of marks will lead to differet calibratio accuracies. If the bad marks (which deviate from their true values more tha others do) are removed, the the calibratio accuracy will be improved. The idea of a subset search method ca be described as follows: suppose that there are calibratio marks i the calibratio date file. As well kow i computer visio, should be greater tha or equal to 7. Let 7 k, where k is a iteger. Let S be a set of marks. The we ca search all subsets of S to determie such a subset which miimizes the calibratio. We estimate how may subsets eed to be cosidered. For a set of size, there are m = k =! k!( k)! subsets of size k, where 0 k. The we have lgm = 1 l10 ( li k ) k li li. For example, let =27, 7 k 27. The the values of lg(.) are betwee 0 ad 7.4. The average of these values is Therefore there are quite a lot of subsets to be cosidered. O the other had, if we ca already calculate from the image that some marks are distorted, the it is better to delete these first. The advatage of this method is to improve calibratio accuracy by searchig ad removig bad marks, but its disadvatage are the computatioal costs D Neighbour Search We ca also cosider eighbors of projected marks i the image plae as possible replacemets, i order to improve the calibratio accuracy. I mathematical terms, assume that there are k (e.g. k = 27) marks. Let r be a positive real umber, W(x 0i,y 0i )={(x,y) Z : x x 0i r, y y 0i r}. S i = {(X i,y i,z i,x i,y i ) : (x i,y i ) W(x 0i,y 0i )}, where Z is the set of itegers. X i, Y i, Z i are the world coordiates of the (i + 1)th mark, (x 0i,y 0i ) are the image coordiates of it, ad i = 0,1,...,k 1. Like the subset search method, usig S i,i = 0,...,k 1, we ca create a umber of calibratio data files ad compute their calibratio s. The, we compare these calibratio s ad choose oe calibratio data file such that the correspodig calibratio is miimal. I defiig W(x 0i,y 0i ), we could also cosider oiteger coordiates so as to obtai more accurate result. The disadvatage of this method is agai its computatioal complexity. If we cosider all 4-adjacet grid poits as possible alteratives, the we have a search space of size 5 k, which already idicates iefficiecy of this approach. 2.3 Least-Squares Error Assume that there are k = x m marks, which meas there are rows ad m colums, where, m 3. For every row, there are m correspodig image coordiates, deoted by P i (x i,y i ), where i = 1,...,m. From this, we ca obtai a least-squares lie, deoted by RL i, where i = 1,...,m. Similarly, for every colum we ca obtai a least-squares lie, deoted by CL j, where j = 1,...,. The we ca compute the itersectio poit of the lie RL i with the the lie CL j, where i = 1,...,m, j = 1,...,. Replace the origial image coordiates by the correspodig itersectio poits. Our experimets show that this method is very effective. 2.4 Sufficiet Image Coordiates I practice two calibratio poits ca produce + 1 image coordiates by dividig the lie betwee these two poits usig poits. Let P i (X i,y i,z i ) be the world coordiates of these two marks, where,2. The correspodig image coordiates of them are deoted by P i (x i,y i ), where,2. The for every positive iteger, we ca compute + 1 calibratio poits ad their correspodig image coordiates as follows: X i = X 1 + X 2 X 1 Y i = Y 1 + Y 2 Y 1 Z i = Z 1 + Z 2 Z 1 x i = x 1 + x 2 x 1 ad y i = y 1 + y 2 y 1 where i = 0,...,. Our experimets show that this method is effective. See Table 1. Palmersto North, November

3 3 A New Calibratio Method Camera calibratio ca be subdivided ito four steps (capture images, pre-process images, prepare data file, ad produce parameters), ad we describe the ew method alog this lie. 2. Next, umber the 16 marks, so as to match world coordiates to image coordiates. We slightly modified 16 marks ad the applied it respectively to the four pre-processed images from the previous step. Step 4. Ru the software [8] to produce the camera parameters file which is eeded, e.g., for object trackig or 3D aalysis. Figure 1: Distortio of a circle s ceter of gravity uder perspective distortio. The ceter of gravity is shifted dowwards as show i the frotal view of the image plae [7]. We use the right had system i defiig the world coordiates. 3D balls are chose as calibratio marks istead of 2D marks, because 2D marks cause perspective distortio (see Fig. 1). Although it is efficiet, but momets of a circle are distorted whe projected oto the image plae. 3D marks ca avoid this problem: a sphere produces a disk i ay viewig directio (see Fig. 2). Assume that Tsai s calibratio method is used. The mai procedures are detailed as follows: Step 1. Capture images of the calibratio object composed of k spheres i measured positios. For example, we used two plaes as the calibratio object. We arraged 16 black 3D marks o each plae such that the distace betwee ay two eighborig marks is 114 mm ad the distace betwee the two plaes is 11 mm. Step 2. Pre-process the images from step 1 by combiig two processes: Figure 2: The top row shows the upper plae captured by left ad right camera respectively. The bottom row shows the lower plae captured by left ad right camera respectively. 4 Experimetal Results 1. cut off the backgroud, ad 2. reduce the oises ad shadows. We achieved this by applyig a suitable threshold to each pixel to reduce the oise. Fially we ca get oiseless images as show i Figures 2. Step 3. Produce a data file which cotais the world coordiates of every calibratio mark ad their correspodig image coordiates. This ca be doe by combiig the followig two processes: 1. For every calibratio mark, fid out its ceter (image coordiates). Figure 3: Calibratio object with plaar markers arraged i three plaes. Our experimets followed the four schemes as described i Sec. 2 for camera calibratio usig 340 Image ad Visio Computig NZ

4 +1 umber of ormalized calibratio marks calibratio Table 1: Relatioship betwee umbers of calibratio marks ad ormalized calibratio s. evaluatio mea stddev maxerr sse method distorted [pix] udistorted [pix] object space [mm] Table 2: Evaluatio of the camera calibratio parameters for the left camera. evaluatio mea stddev maxerr sse method distorted [pix] udistorted [pix] object space [mm] Table 3: Evaluatio of the camera calibratio parameters for the right camera. 3D marks. First we applied the software [8] to the calibratio object with plaar markers show i Fig. 3, ad the ormalized calibratio we obtaied is The followig experimets showed that the method described i Sec. 2.4 is effective for reducig the ormalized calibratio. We choose 8 marks from the XOY plae ad deoted them as P i, where i = 0,1,...,7 (see Fig. 3). We obtaied 4 lie segmets P 0 P 1, P 2 P 3, P 4 P 5 ad P 6 P 7. For every positive iteger, we ca compute +1 calibratio poits ad their correspodig image coordiates from every lie segmet. Table 1 shows the relatioship betwee the umber of calibratio marks ad ormalized calibratio. Table 3 ad 4 illustrate evaluatios of the camera calibratio parameters for left ad right cameras. Now we replaced the calibratio object by a object havig 3D (spherical) marks. From our experimetal results, we foud that usig 3D marks ca improve the calibratio accuracy, but it depeds o the calibratio eviromet. Although 3D marks will ot cause perspective distortio, it may cast shadows uder ouiform lightig situatios. This shifts the ceters of calibratio marks ad affects the fial calibratio accuracy. We used three lights over calibratio marks to reduce shadows, the calibratio s for usig 3D marks ad 2D marks are 4.2mm ad 3.3mm, respectively. Here usig 2D marks was better tha usig 3D marks, because lightig was ot uiform, 3D marks were affected by shadows. The we improved the situatio by puttig 3D marks uder uiform lightig. The calibratio for usig 3D marks ad 2D marks were 2.7mm ad 3.2mm, respectively. The best results we got for usig 3D ad 2D marks were 2.1mm ad 2.9mm, respectively. We also improved the above results by usig the method described i Sec. 2.3, which is the leastsquares method. Accuracies we obtaied are: the image coordiates is less tha 0.51 pixel; udistorted image plae is less tha 0.52 pixel; ormalized calibratio is less tha Coclusios Traditioal calibratio marks are 2D marks. They cause perspective distortio which ofte affects calibratio accuracy. We itroduced 3D marks which ca completely overcome the perspective distortio problem. But usig 3D marks is restricted by eviromets. It requires uiform lightig, so as to have less shadows. It also requires accurate istallatio of 3D marks at the right positio, because ormally 2D marks are arraged by a plotter, ad 3D marks are arraged by hads. From our experimetal results it became clear that 3D marks improved the calibratio if we work uder expected lightig. Aother importat advatage of usig 3D marks is that it is very useful i trackig of movig objects. We also discussed four schemes to improve calibratio accuracy. Amog these schemes, the method of geeratig sufficiet image coordiates ad usig the leastsquares method have bee proved to be efficiet. Our experimetal results show that by usig the leastsquares method, the calibratio accuracy ca be largely improved. Palmersto North, November

5 Refereces [1] E.R. Davies. Machie Visio: Theory Algorithms Practicalities. Academic Press, Lodo, [2] R. Klette, K. Schlüs, ad A. Koscha. Computer Visio Three-Dimesioal Data from Images. Spriger, Sigapore, [3] S. Mawarig. Uited States Patet Applicatio Publicatio, Pub. Number: US 2002/ A1, Pub. Date: July, 25, [4] Tsai Camera Calibratio Software, /www/tsaicode.html [5] J. Weg, P. Cohe, ad M. Heriou. Camera calibratio with distortio models ad accuracy evaluatio. IEEE Tras. Patter Aalysis Machie Itel., 14: , [6] Y.B. Zhag. Calibratio of dyamic stereo ad biocular stereo for large scale objects. Master thesis, The Uiversity of Aucklad, [7] J. Baltes, N. Hildreth, ad Y. M. Li: The all botz robocup team. I Proceedigs of the PRICAI Workshop o RoboCup, Sigapore, November [8] /www/tsaicode.html (last visited: 10th October, 2003) [9] /lstsqr1dcurve.cfm (last visited: 10th October, 2003) 342 Image ad Visio Computig NZ