Refinement of scene epth from stereo camera ego-motion parameters Piotr Skulimowski, Pawel Strumillo An algorithm for refinement of isparity (epth) map from stereoscopic sequences is propose. The metho is base on estimation of ego-motion parameters of a camera system an 3D scene frame-by-frame preiction. Sub-pixel resolution isparity maps were obtaine an no each single frame isparity computations are require. Introuction: Recovering 3-imensional scene structure is one of the most important tasks in computer vision. Applications of the technology capable of seeing in 3-imensions range from virtual reality an robot guiance systems, photogrammetry (remote sensing), enoscopic surgery to electronic travel ais for the blin. Major scene epth sensing technologies are the active methos like: raar, liar an ultrasoun scanners, structure light projecting systems an the passive methos that are preominantly base on stereoscopy [1]. The passive methos are particularly attractive as being least interfering with the environment. In this letter we propose a computational metho for improving epth estimation accuracy for mobile stereoscopic systems uner the constraint of short baseline between the camera optical centres. Stereovision basics: Computational stereo is an imaging metho in which at least two planar projections of a scene are analyse for reconstructing its 3-D structure [1]. For the two-view case the cue use for ientifying epth of a unique scene point is the amount of shift (isparity) between image coorinates at which the scene point is projecte onto image planes of the left an right cameras. For a non-verge camera set-up the isparity is given by xl xr nx, where: x, x L R are the row coorinates of the 1
scene point images in the left an right igital cameras accoringly, x is the horizontal pixel size an n is the amount of shift in pixel units. Then, the epths of a scene point can be compute from Z f B, where f is the focal length of the cameras an B is the baseline between camera optical centres, e.g. for B 8cm, f 3. 5mm, an nx 108. 8m the epth is Z 3. 18m. An absolute epth error for the given stereo rig can be Z estimate from: Z. This error is inversely proportional to the fb baseline between cameras. Hence, the trae-off between system mobility (short baseline) an epth accuracy nees to be resolve. For the stereo rig consiere in the given example, for pixel shifts n 01,,, 3, the estimate epths are, 31. 8m, 15. 9m, 10. 6m, corresponingly. See Fig. illustrating the problem of poor epth resolution ue to short stereo baseline. Sub-pixel epth resolution can be achieve by using interpolation methos which, however, require aitional computations an can give unsatisfactory results for texture-less scene regions [3]. Ego-motion estimation: Let X,Y,Z be a right-hane coorinate system attache to the stereo rig, which is assume to move smoothly in a static environment. 6DoF (egrees of freeom) ego-motion parameters of the rig are efine by two vectors: translational motion vector T rotational motion vector T ω T U V W an. Let a feature point of the environment has instantaneous coorinates P X,Y,Z. Then velocity T V X Y Z of this point is given by [3]: X U Z Y Y V X Z Z W Y Z (1)
Assuming perspective projection of the feature point P X,Y,Z onto image point p x, y, an instantaneous velocity,v plane (be it the right image) can be obtaine from [4,5]: u of this point in image Uf xw u Z Vf yw v Z xy x f f y f f f y xy f x () Equation () relates motion of scene points (relative to the rig) with D motion vector fiel in the image plane. By etecting at least 3 well efine feature points of a scene an tracking them frame-to-frame 6DoF stereo rig ego-motion parameters can be iteratively compute. A fast an robust graient-base metho propose in [6] is use for etecting the feature points (see Fig. 1) an a local block matching technique is use for tracking the features. The ego-motion parameters are compute by solving the normal equation obtaine from the least squares cost function [7]: u x x' v y y' E (3) where: x, y an ', y' x are feature point coorinates ientifie in two image consecutive frames an u,v are substitute from (). Disparity refinement algorithm: The block iagram in Fig. 3 explains the concept of iterative computations leaing to isparity refinement. First, for an initial frame t ense isparity map is calculate an the ego-motion parameters are estimate from motion vectors u,v of the tracke feature points. Then, again using () the next t 1 image frame is preicte. Finally, correction of epth value for each point of the isparity map is compute from (1). To avoi propagation of errors (e.g. ue to occlue objects) the estimate isparity map nees to be verifie against the isparity map compute irectly from the stereo algorithm. This 3
verification, however, nees to be one for every i-th frame only. Hence, consierable savings in computing eman can be achieve. A simple an effective proceure for eliminating error propagation is implemente. If isparity e t estimate for t frame iffers more than 0.5x from the value s t calculate for the same image frame pair from the stereo algorithm, the estimate isparity value e t is correcte accoring to the formula: e e s e t1 t t t, where <1 is a constant value experimentally set to 0.4, for which efficient reuction of error propagations were obtaine on the teste sequence). Experimental results: The propose metho for epth refinement was teste on image sequences taken by in-house stereo rig with geometric parameters as use in the earlier example. The system was mounte onto a tripo enabling 6DoF motion measurements. The tripo was fixe to a platform that was rolle along a 6m long rail (see Fig. 1). Every capture image pair was correcte for geometric istortions an rectifie before ense epth calculations were run by using the stereo block matching technique [1]. The recore sequences consiste of 60 frames taken with 7 frames/sec rate. Table 1 summarizes results of ego-motion estimations for the test sequences. A single frame taken from the refine isparity map sequence estimate accoring to the propose algorithm is epicte in Fig. 4. Note, consierable improvement in the quality of the epth map versus the map that was calculate solely from the stereo-vision algorithm (Fig. ). The contours of iscrete epth layers visible in Fig. have been smoothe out. Conclusion: A proceure for refinement of epth map sequences from camera ego-motion 6DoF parameters was propose. Effectively, the subpixel accuracy of 3D scene reconstruction has be achieve. An avantage of the algorithm is that time consuming the stereovision matching algorithm oes not nee to be run every single frame. Instea, epening on the application, the epth maps can be verifie for every i-th frame. The 4
propose epth refinement metho is particularly suitable for mobile stereovision systems requiring short camera baseline, e.g. for mini-robots. Acknowlegments: This work has been supporte by the Ministry of Eucation an Science of Polan grant No. R0 013 03 in years 007 010. References [1] Brown M.Z., Burschka D., Hager G.D., Avances in Computational Stereo, IEEE Trans. on Pattern Analysis an Machine Intelligence, vol. 5, no. 8, pp. 993 1008, 003. [] Fusiello, A., Trucco, E., an Verri, A.: A compact algorithm for rectification of stereo pairs, Mach. Vis. Appl., 000, 1, (1), pp. 16 [3] Szeliski R., Scharstein D., Symmetric Sub-Pixel Stereo Matching, 7th European Conference on Computer Vision, Copenhagen, Denmark, vol., pp. 55 540, May 00. [4] Bruss A. R., Horn B. K. P., "Passive Navigation,'' Computer Vision, Graphics, an Image Processing, vol. 1, No. 1, pp. 3-0, Jan. 1983. [5] Duric, Z., an Rosenfel, A.: Image sequence stabilization in real time, Real-Time Imaging, 1996,, pp. 71 84 [6] Lepetit V., Fua P., Towars Recognizing Feature Points using Classification Trees, EPFL Technical Report IC/004/74, 004. [7] Skulimowski P., Strumiłło P., Refinement of isparity map sequences from stereo camera ego-motion parameters, ICSES 006 International Conference on Signals an Electronics Systems, Łóź, Polan, September 17-0, 006, Łóź, Polan, pp. 379-38. Authors affiliation: Piotr Skulimowski, Pawel Strumillo (Institute of Electronics, Technical University of Loz, 11/15 Wolczanska Str., 90-94 Loz, Polan) e-mail: piotr.skulimowski@p.loz.pl 5
Fig. 1. The scene use for testing the epth refinement algorithm; note, the scene feature points selecte for tracking Fig.. The epth map compute with pixel accuracy for the scene shown in Fig. 1 (brighter regions correspon to closer objects) Fig. 3. Block iagram explaining the propose metho for egomotion estimation an epth refinement. 6
Fig. 4. A single frame taken from a sequence of refine epth maps obtaine from the propose refinement algorithm (same frame as in Fig. ) Table. 1. Verification of ego-motion parameters estimation (rotation angles are given in egrees) 7