Robust Camera Calibration for an Autonomous Underwater Vehicle

Transcription

1 obust Camera Calibration for an Autonomous Unerwater Vehicle Matthew Bryant, Davi Wettergreen *, Samer Aballah, Alexaner Zelinsky obotic Systems Laboratory Department of Engineering, FEIT Department of Systems Engineering, SISE The Australian National University Canberra ACT 000 Australia Abstract At the Australian National University we are eveloping an autonomous unerwater vehicle for unerwater exploration an inspection. One of our aims is to track the relative position of unerwater targets. This has require the evelopment of a camera calibration system that can overcome the ifficulties of unerwater vision to provie accurate camera parameters. Conventional camera calibration systems etect an then ientify points in an image of a known -D calibration pattern. Point ientification algorithms typically require the full set of calibration points to be etecte to register the target, but this requirement is selom satisfie in unerwater images. We escribe a point ientification algorithm which oes not rely on complete point etection, base upon the inexing of planar invariants calculate from points on a -D calibration pattern. Unerwater experiments have shown our new metho improves the likelihoo of successful calibration by up to 80%, an that our calibration system calculates camera parameters enabling range estimation of targets up to metres away with 95% accuracy. 1 Introuction At the Australian National University we are eveloping an autonomous unerwater vehicle (AUV) name Kambara. We are researching technologies allowing Kambara to search an navigate unerwater environments, using visual guiance base on range estimation an feature tracking techniques. 1.1 Unerwater Stereo Camera Calibration Most range estimation algorithms require knowlege of how pixel coorinates in stereo images correspon to point coorinates in -D space. Camera parameters efine this corresponence, an the proceure of etermining the parameters is known as camera calibration. Camera calibration has been an area of extensive research. Calibration algorithms vary across ifferent applications, but most involve processing images of a FIGUE 1: Kambara s camera suite calibration pattern. The camera parameters are calculate from information extracte from the corresponence between points on the pattern, an pixels in the pattern image [Trucco,1998]. Calibration patterns are require to provie a set of -D points, with a relative geometry known to an accuracy that excees the require accuracy of the vision system. A typical calibration pattern consists of one or two planar gris of rectangles, or boxes, on a contrasting backgroun [Tsai, 1987]. Points are extracte from images of the pattern using ege etection, line fitting, an line intersection techniques. A point ientification algorithm is applie so that each etecte image point can be matche with its corresponing point in -D space. If all the points on the calibration pattern are successfully etecte then this requires a simple point orering algorithm. Unerwater environments present challenges to the reliability an accuracy of calibration algorithms. Ege etection is challenge by the fact that the contrast in unerwater images reuces with epth [eynols, 1998]. Eges in some regions of an image may therefore be misse, an consequently not all points in a calibration pattern image can be reliably etecte. Simple point orering algorithms are therefore impractical in unerwater environments. We have aresse this problem by eveloping a point ientification algorithm which oes not rely on the full set of points being etecte. The algorithm is base on inexing planar projective invariants calculate from points on the calibration pattern. * Now with The obotics Institute, Carnegie Mellon University.

2 1. Kambara s Vision System Kambara s vision harware is comprise of three cameras, shown in Figure 1, an a igitizer. Stereo vision is accomplishe through two Pulnix wie angle lens cameras (TMC-7M), house in moveable waterproof containers mounte on the Kambara frame. A Sony pan-tilt-zoom camera (EVI-D0) is mounte in the upper watertight enclosure, capturing images for Kambara s user-interface. A PXC00 Imagenation framegrabber multiplexes the three camera signals, elivering image frames to the onboar computer. The following section escribes the stanar approach to camera calibration, an the problems pose by unerwater imaging. Section iscusses our new approach to the point ientification problem, an section 4 escribes our experiments evaluating the robustness of the new approach. Section 5 iscusses an experiment testing the unerwater accuracy of our calibration system. Camera Calibration Overview.1 Camera Parameters Camera parameters characterise the mathematical moel use to escribe a camera. Tsai s camera moel [Tsai, 1987] uses two parameter categories: extrinsic parameters efine the position an orientation of the camera relative to a worl reference frame; intrinsic parameters efine the internal projective geometry of the camera, incluing focal length an lens istortion. Appenix A lists an explains these parameters.. Calibration Algorithm Most camera calibration algorithms use a calibration pattern to accurately locate points in space [Trucco, 1998 an the references therein]. These -D point coorinates are matche with their corresponing image pixel coorinates as the basis for an algorithm which calculates the camera parameters. A typical calibration pattern uses two planar gris of boxes on a contrasting backgroun [Tsai, 1987], such as the one shown in Figure. Algorithm 1 lists the stanar algorithm, use with planar box-gri calibration patterns, for fining the parameters of a single camera. The extrinsic parameters of each camera can then be use to erive the stereo extrinsic parameters of the camera suite. 1) Capture an image of the calibration pattern ) Detect points: a) Detect eges b) Fin box eges c) Fit lines to each box ege ) Intersect lines to fin corner points ) Ientify points 4) Calculate parameters ALGOITHM 1: A stanar camera calibration algorithm. The performance of a calibration algorithm is juge by the accuracy of the camera parameters etermine in the parameter calculation step, an the reliability of successful calibration. Both performance characteristics are epenent on the quality of the capture image.. Point Detection Unerwater environments ten to cause errors in ege etection, which propagate through to errors in point etection. Ege etection locates pixels in regions where the image intensity unergoes sharp variations [Trucco, 1998]. Unerwater environments compromise the etection of intensity variations, because the scattering of light by suspene unerwater particles ecreases image contrast [eynols, 1998]. Consequently eges, an therefore corner points, may not be etecte in some regions of an image. The light scattering effect also tens to blur images [eynols, 1998], reucing the sharpness in etecte variations. This makes it ifficult for ege etection algorithms to locate eges accurately. Lines fitte to inaccurately locate eges will eviate from the true box ege, leaing to inaccurately locate corner points..4 Point Ientification Point ientification is the task of ientifying each etecte point in a projecte image of the calibration pattern. If all the corner points are etecte, as is the case in typical calibration environments, then simple point sorting algorithms can be evise to ientify the points on the basis of their vertical an horizontal orering. In unerwater applications the etection of all points cannot be relie upon, an ifferent techniques must be evise. These are challenge by the fact that much of the geometric information relating the points is lost uner projection. In particular, the relative istance between points, the area of a box, an parallelism are not preserve uner projective transformations [Muny an Zisserman, 199]. We have aresse the problem with an ientification scheme base on the inexing of planar projective invariants. Point Ientification using Invariant Inexing We propose a scheme using an inex of planar projective invariants to ientify boxes in images of a calibration pattern. The ientifie boxes can then be use to ientify the box corner points..1 Kambara s Calibration System The calibration system evelope for Kambara is base upon Algorithm 1, an uses the stanar calibration pattern shown in Figure. Both planes of the pattern contain 1 boxes, each with imensions 60mm x 75mm, proviing a total of 96 points [Fitzgeral, 1999]. The calibration pattern images capture by the stereo cameras have a 640 x 480 resolution. A Canny ege etection algorithm [Canny, 1996] is use to extract the eges of each box in the image, followe by orthogonal regression line fitting. Lines are then intersecte to extract points at the corners of each box. The point ientification step of Algorithm 1 was initially implemente with simple point orering algorithms [Fitzgeral, 1999], assuming that all the corner points can be etecte reliably. This assumption was foun to be unrealistic, motivating the evelopment of our new algorithm.

3 Two invariants I 1 an I calculate from boxes B 1 an B can be collectively terme as a box-pair invariant I B1-B, graphically represente in Figure as an arrow from box B 1 to box B. Each possible box-pair in a calibration pattern will have an associate box-pair invariant. Box-pair invariants have the following uniqueness properties: FIGUE : Kambara s calibration pattern. Planar Projective Invariants A projective invariant is an algebraic property of a set of points which remains constant uner projection. Two projective invariants can be calculate for 5 coplanar points x i, i {1...5}: [ x4 x x1 ] [ x5 x x] I 1 = (1a) [ x x x ] [ x x x ] 4 5 [ x4 x x1 ] [ x5 x x1 ] I = (1b) [ x x x ] [ x x x ] 4 1 where the points x i are represente as homogeneous column vectors [othwell,1995]. There are two important collinearity properties of planar projective invariants which affect the choice an orering of planar points [othwell, 1995]: 1) If any points are collinear, then either I 1 or I is singular; ) If both x 1 an x are collinear with any of the other points, then both I 1 an I are singular. These properties emonstrate the invariants epenence on point orering. The secon property leas to a constraint where any set of 5 points must be chosen such that x 1 an x are not collinear with a thir point.. Ientifying Box-Pairs with Planar Projective Invariants The invariants given in (1a) an (1b) can be use to ientify pairs of boxes. Consier two boxes B 1 an B : we can take x 1 an x from B 1, an the remaining points from B to calculate I 1 an I. The selection of points use in our algorithm is shown in Figure, with the point orer chosen to avoi having both I 1 an I singular for any possible combination of boxes. 5 a) b) FIGUE : a) The corners selecte for invariant calculations; b) Box-pairs ientical by translation share the same invariant: I 0- =I -5 =I 6-8 =I ) For any two boxes B 1 an B, I B1-B I B-B1 ; ) Box pairs ientical uner translation share the same invariant. Figure (b) illustrates these uniqueness properties. Numbering the boxes 0 through 11 for one plane of the calibration pattern, we see box-pairs 0-, -5, 6-8, an 9-11 all share the same box-pair invariant..4 Invariant Inexing A box-pair invariant inex can be constructe to ai the ientification of box-pairs in images of the calibration pattern. Given a box-pair invariant the inex returns a list of the corresponing box-pairs. The inex has an entry for each unique box-pair invariant. For a 4 x gri of boxes there are 4 unique invariants associate with 1 box-pair combinations. The invariants in the inex are calculate from precise measurements of the calibration pattern. The invariants use to key the inex are calculate from images of the calibration pattern, an will always iffer slightly from the inex invariants. There are two main causes of this ifference: 1) Projective invariants are base on an assumption of pinhole projection. aial lens istortion in the cameras means the pinhole moel is not an accurate moel for projection; ) The ege etection an line fitting of boxes has inaccuracies which propagate through to box corner point calculations, which in turn leas to errors in invariant calculations. This isparity requires the use of invariant tolerances. When searching the inex, a match is foun if the image-base invariant falls within the tolerance range of the inex invariants. If the tolerance is too small then there will be many false negatives, resulting in many points not being ientifie. There are more serious consequences if the tolerance is too big: false positives will be mae, corrupting the calculation of the camera parameters. A tolerance of ±0% was empirically etermine to work reliably without false positives, an with only a small number of false negatives..5 Inex Search Algorithm The invariant inex is use to ientify a set of planar boxes. Keying the invariant inex with an invariant calculate from two of these boxes will return a list of possible box-pairs. Only one of these is the true box-pair. Calculating invariants between the other boxes can help eliminate possibilities, until a unique combination of boxes has been ientifie. This is the basis for the invariant inexing algorithm state in Algorithm. The algorithm uses a search structure, with branches of the structure representing possible box combination scenarios. As more invariants are calculate, branches from the search structure are prune as possible scenarios are eliminate, until one branch is left which

4 ientifies the boxes. The inex searching algorithm is illustrate in Figure 5. In this example only 4 boxes have been etecte from a planar gri: B 1, B, B,anB 4. The box-pair invariant I B1-B between boxes B 1 an B is calculate an use to search the invariant inex. A match is foun, fining B 1 an B to be one of possible box-pairs. A search structure is create by allocating a branch to represent each of the scenarios: i) B 1 is, ii) B 1 is 6, an iii) B 1 is 9. better the accuracy of that parameter estimate. Fortunately this is not a serious problem unerwater, since the air-water interface refracts light so as to reuce lens istortion. With Kambara's vision system it was foun the calibration pattern can be brought close enough to occupy most of the vertical fiel of view (FOV) of each camera. 1) Begin with a list of n unientifie boxes, B 1, B...B n ) Calculate the invariant I B1-B from B 1 to B. ) Key the invariant inex with I B1-B to fin the corresponing list of box-pairs. 4) Create a search structure with one branch allocate to each of the box pairs foun from the inex. 5) For the remaining boxes B i, i {...n}: a) Calculate the invariant I B1-Bi b) Use I B1-Bi to look up the invariant inex c) For each box-pair BP j, j {1...k}, foun in the inex: i) Search the structure for a branch having the same base box as BP j. ii) If such a branch is foun, then a BP j to it ) Prune any branch which oesn't have a base box corresponing to any of the box-pairs. 6) Succee if only 1 branch remains in the search structure, otherwise the boxes B 1...B n are insufficient for ientification. ALGOITHM : Ientifying boxes using a search structure an an invariant inex. Next the invariant I B1-B between boxes B 1 an B is calculate. Searching the invariant inex fins that B 1 an B coul be one of 6 possible box-pairs. Only two of them share the same base boxes with branches of the search tree, namely box-pairs 6-0 an 9-. These are ae to the corresponing branches. No possible box-pairs are foun consistent with scenario i), so it is prune. Next the invariant I B1-B4 between boxes B 1 an B 4 is calculate. Searching the invariant inex fins that B 1 an B 4 coul be one of 6 possible box-pairs. Only 1 box-pair shares the same base box with a branch in the search tree, namely 6-10, an this is ae to the corresponing branch. None of the box-pairs are foun to be consistent with scenario iii), so this branch is prune. Only one branch remains, revealing the true ientity of the boxes: B 1 is 6, B is 5, B is 0, an B 4 is Performance Limitations A requirement that must be satisfie for the invariant algorithm to work is that at least one box from each outer bounary of the planar gri must be etecte. The number of etecte boxes require for ientification is therefore epenent on which boxes are etecte. A box on two opposite gri corners is sufficient for ientification, since each corner box shares two bounaries. The maximum numberofboxesrequireina4xgriis10. aial istortion prevents the target being place too close to the cameras, as the true invariants increasingly eviate from the pin-hole moel invariants. This conflicts with the aim of accurately characterising raial istortion, because the more istortion capture by the image points the FIGUE 5: An example illustrating Algorithm. Boxes B 1, B, B an B 4 have been etecte in the image of a calibration pattern, but their ientity is unknown. The invariant inex is use together with the search tree to ientify the boxes. The search tree contains branches representing ifferent possible ientities. Arrows represent box-pair invariants, while a cross represents a prune branch of the search structure. 4 Calibration obustness Evaluation The performance of the calibration system is juge on both the reliability of the system an the accuracy of the calculate camera parameters. This section escribes the experimental evaluation of our calibration system s reliability.

5 4.1 Experimental Proceure The performance of the invariant algorithm was teste by comparing the reliability of Kambara's calibration system using a) the invariant inexing algorithm, an b) simple point orering algorithms reliant on all 96 corner points being etecte. Calibration testing use unerwater stereo image sets of the calibration pattern. The image sets were collecte by placing Kambara unerwater, an capturing images from the stereo cameras as the calibration pattern was move about the stereo FOV. The stereo camera pair, apart from being slightly verge to enable a close stereo range, were arbitrarily oriente. The calibration pattern was oriente so as to be approximately horizontal in the FOV of each camera. Not all the capture image sets were fit for calibration. Image sets were iscare if a) both planes of the calibration pattern was not seen clearly in the FOV of both cameras; or b) the calibration pattern was more than metres away from the cameras if the calibration pattern is further away than this, then each projecte box ege has too few pixels for accurate line fitting. A total of 4 image sets satisfying these requirements were use for testing our calibration technique. Each image set was use twice for calibration, once each for the point orering an invariant inexing algorithms. 4. esults We foun the invariant inexing algorithm to be far superior than simple point orering. The results of comparison are summarise in Table 1. Although the point orering algorithm le to the successful calibration of one of the stereo cameras for 40% of the image sets, calibration of both cameras never occurre. This result emphasises the ifficulty of etecting all 96 corner points in an unerwater environment. The invariant inexing algorithm le to the successful calibration of both cameras for 80% of the image sets. Point Ientification Algorithm % of images sets for which at least 1 camera calibrate successfully % of images sets for which BOTH cameras calibrate successfully Point orering 40% 0% Invariant inexing 81% 80% TABLE 1: Evaluation of calibration system reliability with point orering an invariant inexing point ientification algorithms. 5 Calibration Accuracy Evaluation Unerwater environments present challenges to the accuracy of point etection, an consequently the accuracy of camera calibration. This section iscusses the challenges of evaluating calibration accuracy, an outlines a methoology which evaluates accuracy using -D object imension estimation. 5.1 Difficulties in Parameter Accuracy Evaluation Evaluation of the calibration system accuracy is challenging because of the ifficulty in obtaining groun truth for comparison. A camera s intrinsic parameters are epenent on many factors, incluing the calibration environment, so it is impossible for a camera manufacturer to provie comprehensive parameter specifications. A camera s extrinsic parameters can in principle be compare with measurements mae between camera an worl reference frames. In practice, however, such measurements are inaccurate an therefore inappropriate for comparison, since a) the origin of the camera reference frame is locate within the camera, an therefore inaccessible to ruler measurements, an b) it is impossible to accurately locate the origin of a camera reference frame. Another approach might be to use the accuracy of range estimation to infer the accuracy of the camera parameters. ange estimation estimates the vector from a camera reference frame to a point in -D space. Unfortunately inaccessibility of the camera reference frame again prevents meaningful comparison with measure istances. 5. Experimental Methoology We have evise an experimental approach which compares the estimate length of objects in Kambara s environment with accurate ruler measurements. The length estimates are foun by firstly using range estimation to etermine the range vectors to two points in space, an then using simple vector arithmetic to fin the istance between the two points. If length estimates are accurate then it can be inferre that range vectors are accurate, which in turn infers that the calculate camera parameters are accurate. The range estimation algorithm use for this experiment is liste in Appenix B. 5. Experimental Proceure Kambara was place unerwater together with the calibration pattern, an the stereo cameras were calibrate. The calibration pattern was then use to provie pairs of points separate by precisely known istances. Estimates of point-pair isplacements were taken with the calibration pattern approximately 1, an metres away from Kambara. At each of these istances the pattern was move about the FOV of each camera, an at each position roughly 8 istance measurements between point pairs were calculate. 5.4 esults The accuracy of length estimation at each istance was calculate as a percentage error of the true (measure) length. The average an maximum percentage errors for eachistancearelisteintable. Length estimates were foun to have a maximum mean error of 5%, inicating that the calculate camera parameters are sufficiently accurate for range estimation of targets up to metres away. It was also foun that range estimates improve as the target approaches closer to the AUV, which is consistent with our intuition that small errors in a range vector's orientation will amplify with istance. Approximate istance between AUV an target Mean Error % St Dev % Max Error % 1 metre 1.4 % 1% 6% metre.6 %.7% 15% metre 5.0 % 5.% 0% TABLE : Evaluation of calibration system accuracy

6 6 Conclusion We have evelope a new point ientification algorithm, base on planar projective invariant inexing. Experimental evaluation has shown the algorithm to be up to 80% more reliable than simple point orering algorithms. Further experiments have foun Kambara's calibration system, base on our invariant inexing algorithm, to be sufficiently accurate for range estimation applications with target istances up to metres. There is scope for improvement in the reliability of unerwater calibration. Fining an optimum invariant tolerancing scheme woul help reuce inex false negatives, an therefore increase the calibration success rate. obustness coul be further enhance by eveloping techniques allowing the calibration pattern to be hel at arbitrary orientations with respect to the camera suite. In conclusion, we believe that we have evelope an teste a camera calibration scheme suitable for unerwater computer vision applications. Acknowlegments We wish to thank all current an past members of the Kambara team for their tireless support an encouragement. We are especially grateful to Ian Fitzgeral for eveloping the original Kambara calibration system. A Camera Parameters The intrinsic parameters of a camera consist of: f - focal length of the camera; K 1 - raial lens istortion coefficient; (C x, C y ) - pixel coorinates of the centre of lens istortion; s x - horizontal scale factor; ( x, y )- the "centre-to-centre" istance between ajacent CCD sensor elements in the X an Y irections; N cx - the number of CCD sensors in the X irection; N fx - the number of pixels sample in the X irection. The stereo extrinsic parameters efine the mapping between the camera reference frames of the left an right cameras (enote {L} an {}). They consist of a translation L L L L P = p p p an rotation matrix L. vector [ ] T x y z B ange Estimation Algorithm The following range estimation algorithm is aapte from the work of [Horn, 1986] an [Tsai, 1987]. The algorithm fins range vectors from the reference frames of the left an L L L L P = p p p an right camera to a point P: [ ] T x y z P = [ p p p ] T x y z, as illustrate in Figure 6. There are twomainstepsinvolve: 1) Use the intrinsic camera parameters for each camera to map the target coorinates ( X f, Y f ) observe in the capture image, to unistorte coorinates ( x ', y ' ) corresponing to the pinhole camera moel [Tsai, 1987]: x ' = K X + K X Y + X y ' = K Y + K Y X + Y 1 (a) (b) where ( X X Y FIGUE 6: ange vectors from the left an right camera frames to a point P., Y ) are foun from ( X f, Y f )by: = X C s N N (a) ( ( f x ) x fx ) cx x ( Yf Cy ) y = (b) ) The unistorte image coorinates in the left an right stereo images, ( x ' L, y 'L )an( x ', y ' ), the focal lengths of each camera, an the stereo extrinsic parameters, are substitute into the following equation, erive from the frame transformations between the stereo camera an the target reference frames [Horn, 1996]: ' ' x f xl f L L ' ' L y f pz = y L f L pz (4) 1 1 This vector equation is solve to fin L pz an p z.wethen substitute into the following equations to fin the range vectors from the left an right cameras, L P an P : L P P T ' ' xl y L L = 1 f L f L T ' ' x y = 1 f f p p z z (5a) (5b) eferences [Canny, 1996] Canny, J. A Computational Approach to Ege Detection. IEEETrans.onPAMI,Vol.8,No.6, pp [Fitzgeral, 1999] Fitzgeral, I. A Vision System for an Autonomous Unerwater Vehicle. Department of Engineering ANU, [Horn, 1996] Horn, B.K.P. obot Vision. The MIT Press Cambrige, Massachusetts, [Muny an Zisserman, 199] Muny, J. an Zisserman, A. Geometric Invariance in Computer Vision. MIT Press, 199. [eynols, 1998] eynols, J. Autonomous Unerwater Vehicle: Vision System. Department of Engineering ANU, [othwell, 1995] othwell, C. A. Object ecognition Through Invariant Inexing. Oxfor University Press, New York, [Trucco, 1998] Trucco, E. an Verri, A. Introuctory Techniques for -D Computer Vision. Prentice-Hall, [Tsai, 1987] Tsai,. Y. A Versatile Camera Calibration Technique for High Accuracy D Machine Vision Metrology Using Off-the-shelf TV Cameras an Lenses. IEEE Trans. on obotics an Automation,Vol. A-, No. 4, pp. 44, 1987.