Camera Calibration Utility Description

Transcription

1 Camera Calibration Utility Description Robert Bryll, Xinfeng Ma, Francis Quek Vision Interfaces and Systems Laboratory The university of Illinois at Chicago April 6, Introduction To calibrate our cameras for stereo viewing we use Roger Y. Tsai s versatile camera calibration algorithm described in [1]. The problem with calibrating two cameras for stereo viewing is that to calibrate them it is necessary to record a calibration target with points of known coordinates and then find the image coordinates of calibration points on images from both cameras, put them in files in appropriate order and pass them to the calibration routines. Tsai s algorithm requires at least 11 calibration points, but the normally used number is between 20 and 60. Picking those points by hand to create input files for calibration rooutines is tedious and time consuming. We make the proces of finding calibration points in frames captured from both cameras semiautomatic. 2 Solution The calibration target (a plywood box with two walls put at an angle of 120 degrees to one another) is painted white and all the calibration points (of known coordinates) are marked with black 3/4 circles. The size of the calibration target approximates the average volume of the human gesture space. The calibration target is presented in Figure 1. Figure 2 shows how the world coordinate system is positioned with respect to the calibration target. During the calibration the box should fill approx. 1/3 rd of the frame area. Black calibration points are detected semi-automatically by the following steps: 1. Threshold the input images to detect dark areas (thresholds are adjusted by hand). 2. Find connected components in the image, label the detected regions. 3. Run size filter over the detected regions. Since the calibration points are relatively small (around pixels according to our tests), this step filters out many unwanted regions. 4. Remove false positives by hand. 5. Compute centers of gravity of the final set of regions to obtain more precise locations of the calibration points. 1

2 Figure 1: Stereo camera calibration target used in our experiments. 2.1 Camera Calibration Utility (CCU) Figure 3 shows the Camera Calibration Utility interface that allows the user to perform the steps described above semi-automatically. The following sections describe the steps necessary to perform the calibration point detection and to save the resulting files Input Files There are two input files for the calbration (one image from each camera). They should both be in the raw data grayscale.pgm format, both have the same dimensions (close to pixels). The names of the files should follow a simple convention: the filename of the lower image is equal to the name of the upper image with extension.2. So, for example, if the upper image file is calib1.pgm, the lower image file should be calib1.pgm.2. Both files are opened by pressing the Load Images button and selecting the name of the upper image only Thresholding After opening the image files, they are displayed in the window as in the figure 3. Then the user has to adjust the thresholds for both images, so that the calibration points are clearly defined on the faces of the calibration target on both images. The thresholds are adjusted by moving the sliders in the Thresholds part of the interface window. When sliders are moved, the corresponding image changes, showing the result of the thresholding. The faces of the calibration target should be black 2

3 X Z Y X Figure 2: Position of the World Coordinate System with repect to the calibration target. and calibration points should be represented by white dots. The default values of the thresholds work in some cases, but usually they have to be adjusted (or at least the results of the thresholding have to be checked, by clicking on the sliders). It is possible to display the original images at any time by pressing the Show Originals button. It is also possible to adjust size filters ro the calibration points, but in our experinments it was never necessary. It may be necessary if the calibration target is very close or very far away from the cameras Detecting Calibration Points After the thresholds have been set, the user should press the Detect Points button to perform the actual detection. The program filters the image with 55 Gaussian filter, then thresholds it according to the adjusted threshold values, finds connected components and runs size filter over them. Finally, it computes the ceter of gravity of each accepted region and displays the ceners positions on the original images. The accepted regions are represented by red crosses with corresponding numbers. After this stage the regions are unsorted and there is no correlation between the regions on both images Removing False Positives After the detection stage there usually are some false positives that have to be removed. The point can be removed by left-clicking on the red cross representing the calibration point (another click will turn the point back on). All false positives must be removed, and moreover, the sets of points on both calibration images have to be identical (the same number of points, same positions on the target!). For example, in figure 3 we can see that on both images the leftmost column of calibration points is unsused. If on the upper image there are calibration points in this column, and on the lower one there are no calibration points there (because of false negatives), the calibration points have to be removed from the upper image to make both sets identical. 3

4 Figure 3: Camera Calibration Utility Interface. 4

5 2.1.5 Setting the Points on the Calibration Points Panel and Sorting the Calibration Points In order the correspondence between two sets of points to be resolved correctly, the sets of calibration points have to be reflected on the Calibration Points Panel. Calibration Points Panel contains two sets of 25 buttons, each set represents one face of the calibration target. Initially all buttons contain are checked (contain letter x), whicgh means that all 50 points are selected. However, if there are false negatives on the calibration images, the user has to uncheck the necessary calibration points to reflect the actual detected state. Figure 3 shows the situation in which the leftmost column of the calibration points is unchecked, and there are four unused (undetected) calibration points in the center of the calibration target. Setting the Calibration Points Panel is very important for sorting the calibration points on both images so that there is exact correspondence between the two images. The sorting is performed as follows: 1. The calibration points array for each image is first sorted in place with respect to the image X coordinate, so that the numbers of points are increasing in columns starting from the left. The assumption here is that the target is relatively vertical, that is there is no overlap between the calibration points columns in the X direction. If the target in the image is very far from vertical, the algorithm will fail (because it won t be able to distinguish the columns if there is overlap between the points belonging to different columns in the X direction). However, such situation didn t occur in our experiments, so we think the assumption is safe. 2. The calibration points array for each image is then piecewise sorted in place using the information from the Calibration Points Panel. The pieces sorted correspond to the columns in the calibration target and sorting is done with respect to the image Y coordinate (so now the points are sorted only in columns). That s why the correct information from the Calibration Points Panel is crucial for the algorithm (because the numbers of points in columns do not have to be equal). As the result, the points are ordered starting from the upper point in the leftmost column, and the numbers increase when going down in columns and then to the right between columns. The sortingis done after setting the CalibrationPointsPaneland pressing the SORT POINTS button. The program shows the resulting order (calibration points with their numbers). The situation after sorting is presented in figure 3. After sorting there is exact correspondence between the calibration points on both images and their coordinates can be saved for further processing (using Xinfeng s calibration interface as a wrapper over the Tsai s routines) Saving the Calibration Files After sorting the calibration points, they can be saved into calibration files (simple text files) using two buttons in the lower left corner of the interface window. The Save World & Image Coords button saves 5 floating point numbers for each calibration point: first the three world coordinates (x,y,z) of each detected point and then its two image coordinates (computed with subpixel accuracy as the regions center of gravity). The Save Image Coords Only button saves only the two image coordinates for each calibration point on each calibration image. 3 Naming Conventions for Files The names of saved calibration files follow simple naming conventions. We will show them by example. If the fupper calibration image is called calib1.pgm, the lower image has to be called calib1.pgm.2. Then the names of calibration files for both images will be as follows: 5

6 calib1.pgm.world.dat - the file with world and image coordinates for the upper image calib1.pgm.2.world.dat - the file with world and image coordinates for the lower image calib.pgm.image.dat - the file with only image coordinates for the upper image calib.pgm.2.image.dat - the file with only image coordinates for the lower image The files can then be used (together with the camera data files) by the Xinfeng s camera calibration interface built over the Tsai s calibration routines. 4 Accuracy Tests We wrote a simple program analyzing the output from the world data files after 3D triangulation and comparing it to the original world coordinates. We performed the accuracy tests by first calibrating the camera using a set of calibration pictures and then using the same pictures (image coordinates only) to calculate the real world coordinates. Comparing the calculated world coordinates with the original calibration world coordinates yielded the calibration errors in 3 dimensions. Here are the results of our tests (all errors in mm): 1. Baseline Length approximately 1953 mm; 38 calibration points: Average error in X: mm Maximal error in X: mm Average error in Y: mm Maximal error in Y: mm Average error in Z: mm Maximal error in Z: mm 2. Baseline Length Approximately 1198 mm; 43 calibration points: Average error in X: mm Maximal error in X: mm Average error in Y: mm Maximal error in Y: mm Average error in Z: mm Maximal error in Z: mm 3. Baseline Length Approximately 1186 mm; 39 calibration points: Average error in X: mm Maximal error in X: mm Average error in Y: mm Maximal error in Y: mm Average error in Z: mm Maximal error in Z: mm It can easily be observed that the longer baseline (point 1 above) gives higher accuracy of triangulation. However, even for the shorter baselines the maximal triangulation error never exceeds 5mm, which is adequate for our 3D gesture experiments. 6

7 References [1] R.Y. Tsai, A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal of Robotics and Automation, vol. RA-3, pp ,