Handy Rangefinder for Active Robot Vision

Transcription

1 Handy Rangefinder for Active Robot Vision Kazuyuki Hattori Yukio Sato Department of Electrical and Computer Engineering Nagoya Institute of Technology Showa, Nagoya 466, Japan Abstract A compact and high-speed rangefinding system which is applicable to active robot vision is described. The principle of depth measurement is based on the space encoding method. Spatial coded patterns of light are generated by a semiconductor laser. A slit-ray of the laser is scanned by a rotating polygonal mirror and switched according to temporal switching patterns. Each pattern of light is generated, and its reflected image is taken by a CCD camera, in one image frame with no lag time. A 512 x 256 range map is obtained continuously every 0.2 seconds. I. Introduction An active robot vision system collects a lot of intensity images for a 3-D object, shifting its viewpoint and view direction in the working space. If the system equips a rangefinder instead of an ordinary video camera, it can collect direct 3-D shape information about the object taking 2 1/2-D range maps from various viewpoints. The information that the rangefinder can obtain is including a position and pose in 3D space as well as the shape for the object. Such a rangefinder for an active robot vision requires it to be handy and instantaneous. That is, it should be small and light enough to be installed at the tip of a manipulator, and the exposure time to get range maps should be short enough for the robot to work rapidly. Many rangefinding methods have been proposed, but the active illumination with a slit-ray scanning is one of the most practical methods to get depths of 3-D objects. If we do not use ordinary video cameras but special sensing devices for rangefinding [l] that has parallel range sensors with independent memories, we can get a range map within 1 millisecond. However, so far, the resolution is not highly satisfactory (32 x 28 rangepic [2]), and it has not been commercially issued yet. Using a video camera, as an image input device, is convenient even if it takes longer time to capture images because once some favorable thresholding technique is applied to intensity images, reflections of the illumination can be extracted stably, regardless of the reflection property of the object surface. Furthermore, we can simultaneously analyze both an intensity image and a range map obtained by a single camera. The space encoding technique is one of the best methods to obtain range maps if an ordinary CCD camera is used as an image input device. In the space encoding method, the measurement time mostly depends on the light pattern generation tec:hnique. There have been many attempts to generate pattems of light [3j - [7] but using liquid crystal shutters rnust be the most practical and successful means [3][6]. The istripes of liquid crystal shutters which are installed in a slide projector are switched independently and yield pattems of light rapidly. The piroblem with such a type of light pattern generation lies on the heat which the projector emits, as well as the size and the weight of the projector. A semiconductor laser must be ithe best ffor the illumination source because it involves an extra high optical power with less heat, even though it is extremely small and light. Furthermore, the light can be switched in a snap, as it is well-known as an optical communication device. Such advantages of the semiconductor lead us to realize a handy and instantaneous rangefinder. We have proposed a new type of rangefinding system called the Cubicscope [81. The Cubicscope equips synchronously controlled a CCD camera, a semiconductor laser, and a scalnning mirror and obtains range images based on the space encoding method. In this paper, we have proposed an improved practical prototype that installs a polygon scanning mirror and a specially coordinated hardware system. Such improvements have come to make us realized the Cubicscope is able to capture preferable range images successively and quickly at less than 0.2 seconds per frame. II. Rangefinding by Space Encoding Method In the space encoding method, the illumination space is divided into wedge-like regions, which are equivalent to planes of a slit-ray. Each region is encoded by binary numbers with illluminating light patterns. (For more details, see reference [8j). For each pixel, the slit-ray is decoded from plural coded images, and the depth of the corresponding point on the object surface is calculated based on the triangular formula. Such depth calculations for every pixel on the entire image plane completes the range image for the scene. As 'we see, only N coded image frames are necessary to identify 2N - B slit-rays; this reduces the number of image IEEE International Conference on Robotics and Automation /95 $ IEEE

2 frames to be taken and sharply saves exposure time. It is apparent that the highest exposure speed will be attained if the pattern of light is generated within the period of oneframe, the camera captures the reflection image simultaneously, and no lag time is wasted between image frames. In our rangefinder, the beam of the semiconductor laser is expanded vertically to form a slit-ray, and it is scanned horizontally by a scanning mirror, e.g., a galvano mirror [8]. As we scan the slit-ray and switch the semiconductor laser to a high frequency (generally higher than hundreds khz) during one video field, we can generate an arbitrary stripe pattern of light by temporal signal switching. The CCD camera grabs this stripe pattern in the first video field and outputs in the next video field. If we use fixed spatial pattern shutters like in [6], it is not easy to adaptively modify the stripe patterns to object size, image range, or Fig. 1 polygon mirror (1 2 faces) semiconductor & Light pattern generation with a polygon mirror stripe pitches as well. However, the computer controlled temporal switching easily generates arbitrary spatial stripe patterns. And furthermore, the focusing is improved by using the laser, instead of a slide projector. Obviously, all the signals have to be controlled synchronously to generate patterns: the laser switching signal, the angle control signal of scanning mirror, and the vertical synchronous signal of the CCD camera. This scanning and switching technique potentially allows the most rapid pattern generation of light; a pattern of light is generated, and the reflected image is taken within one image frame with no lag time. As mentioned above, we used a galvano mirror for scanning the laser slit-ray. A galvano mirror can be simply controlled by analog signal, but due to the nonlinearity between the mirror angle and signal, a precise calibration is needed to acquire accurate slit-ray planar equations. A polygon mirror must be the preference to adopt for the laser scan because of the stably controlled scanning speed and the wide scan angle. Therefore, the slit-ray projection angle can be accurately derived for the entire scan range. Fig.1 shows the construction of the pattern illumination with a polygon mirror. However, the polygon mirror control is not so stable when the rotating speed is not sufficiently high, e.g., 1500 rpm or more. This speed is too fast to yield a light pattern within 1/60 seconds; 200 rpm must be suitable with an 18- face polygon mirror for a 40 degree view range. The higher scan produces the less reflected intensities on the image. To solve this problem, we project an identical light pattern many times with successive mirror faces during the image field. This operation results in obtaining higher intensity images than that with a single projection. This pattern generation is achieved by severely synchronizing control signals. In our system, the polygonal mirror has 12 faces, and it is driven by a servo motor at 1800 rpm. Six mirror faces passes during a video field. The rotation speed and phase of the mirror must be controlled synchronously with the video signal. Fig. 2 shows the timing of each control signals to generate the pattern light using this polygon mirror. Vertical (I/[. SYNC U I 6 scans mirror If I signa, camera output U U u u uuu U uuuuuuu U uu U U U uuuuuuuu U U uuuuu UUUUL - Fig. 2 Control signals Fig.3 Cubicscope

3 111. Configuration of Rangefinder Cubicscope Our rangefinder is indicated in Fig. 3. It involves a 2/3 inch CCD camera, 30 mw semiconductor laser, and a polygon mirror installed a servo motor. The size is small (W: 180 mm, D: 85 mm, H: 55 mm) and the weight is light (950 g). This rangefinder is so compact that it can be installed at the tip of a robot manipulator and is controlled in the 3-D space to quickly change its view point or direction. The CCD camera equips a optical filter which cuts off the light except for red wave length. Since the semiconductor laser used here is red (690 nm), the camera stably detects the reflections of the laser in an ordinary luminous circumstance. PO ygon (.9?p semiconductor Drive main controller 1 (NEC PC-9801) Fig. 4 Block diagram of controller I converter, a thresholding circuit, and frame buffers and produces trigger pulses at every 1/60 sec to the computer and the camera controller, and (0 computer (NEC PC-9801) that observes signal timing of the whole system. Fig. 2 shows the timing chart of the signals of respective units. As Fig. 4 schematically represents the block diagram of the control circuit.. The circuit consists of six major units as follows: (a) mirror driver that yields motor drive pulses, (b) laser driver which supplies power with a switch input, (c) drive controller that receives control timing and generates drive signals for the mirror and the laser, (d) camera controller that generates all synchronous signals for the camera, (e) image decoder that involves an image A/D indicated in the figure, while the camera shutter is open, the slit-ray radiates the scene in first video field (1/60 sec), and the camera transmits the video signal in the next video field (1/60 sec), In the image decoder, input images are transformed to slit-images with some thresholding technique. The intensity levels in the blnary image must stably indicate whether the slit-ray reflects onto the image plane. It is easy to use some fixed threshold level for the whole image to binarize input images, but the thresholded images are vulnerable to the reflection property of the object surface. In our rangefinder, the system is designed to adjust the threshold levels adaptively for every pixel. We can apply three methods with our system: (a) fixed threshold method, (b) averaged threshold methlod, and (c) complementary pattern method. The measuremlent time using each method are (a) 8/60 sec, (b) 10/60 sec, (c) 16/60 sec, These times depends on the number of necessary pattern images. For more details about these method, SI= reference [8]. video signal from camera 0 frame memory 8-bit data bus + I-bit data bus Fig.5 Block diagram of image decoder 1425

4 The complementary method is the best method to obtain invariable binary images although it takes a long time to expose. And the averaged threshold method might be practical to adopt for usual industrial use. In order to apply all of these thresholding methods, the image decoder equips an A/D converter, two 8-bit frame buffers, and a comparator, as shown in the image decoder of Fig. 5. We must calculate the range value on each pixel from the space coded image. The range value is obtained by a triangular formula. If the major axis of the laser slit-ray and the y axis of the camera's image plane are precisely parallel each other, the range value z on each pixel is calculated by the following equation: J.1 Z= x + J tan 8 (1 1 where x is horizontal coordinate of the pixel in the image plane and 8 is light projection angle. Obviously, focal lengthfand the distance between the camera and the mirror I are the optical parameters previously determined before measurement by using some calibration technique. As shown by this equation (l), the range calculation of each pixel is the floating point processing. It needs a lot of computational power if we get a range map in real time when using software calculation. To resolve this problem, we developed special hardware which achieves real time range map generation using a hardware calculation. Equation (1) has two discrete variables, x and 8. Parametersf and I are calibrated constant values. Depths of the range image are obtained by striking out the look-up-table with two inputs, x and 8. Since angle 8 is determined by the space code and coordinate x is derived by the horizontal coordinate of each pixel on the frame memory. The range generator indicated in Fig.5 makes above look-up-table process. The sequence of this process is follows : the binary pattern images are copied to the space code buffer. The space code each pixel is read out one by one. X coordinate corresponding to the space code is generated by the X-coord generator. Both space code and x coordinate are input to the range table buffer and images depths are struck out. We can proceed this sequence in parallel with obtaining and binarizing pattern images. It takes 0.2 seconds or less for getting range values. IV. Experimental Results and Conclusions In Fig. 6, the pattern projected images for wooden blocks are shown. The gray code is used for the pattern encoding. Fig. 7 shows the range map for the measured object, where the brighter intensity is depicted closer to the camera. The resolution of the range map is 512 x 256, and the relative accuracy for the depth measurement is less than 1 %, which is comparable with other measurement strategies based on the triangulation formula. In Fig. 8, some other experimental results are presented. We should note about the lens distortion when describing measurement accuracy. The lens we used here is a wide angle one, and it has at the most 3 % distortion around the edge part of the obtained image. This distortion obviously badly influences the measurement accuracy. Therefore, we calibrated this lens distortion using distortion parameters supplied from the lens maker. Fig. 6 Projected pattern images Fig. 7 Obtained range map

5 (a) doll (left: intensity image, right: range image) (b) pile of electric parts (left: intensity image, right:. range image) Fig.8 Measurement result If the objects in the scene to be observed or the rangefinder itself has movement, the measurement error become very large because our rangefinder takes at least 8/60 second to obtain complete pattern images which is necessary to achieve one space encoding measurement. This measurement time is short enough to obtain static scene s range maps or very little movement of objects. On the other hand, in the scene which has movement and vibration of the objects to be ob:served or rangefinder to observe, for example mobile application, the measurement error would become large. In many robotics applications, however, the short stop of the rangefinder is permitted. The measurement time 8/60 sec is very short for such stop of robotics, and then our rangefinder Can achieve accurate space encoding in the

6 very short stop period. In such applications, we believe that our rangefinder has the capability to be a vision sensor. As we have mentioned in this paper, our rangefinder, the Cubicscope, is downsized and instantaneous to get range maps with new image processing hardware. High-resolution range maps (512 x 256 rangepic) are captured within 0.2 seconds with 1% accuracy. It is practically compact enough and it can be applied to an active robot vision, in which it recognizes the 3-D scene taking 2 1/2-D range maps and 2-D intensity images from several view points or directions. Acknowledgements The authors would thank Mr. Shibata and engineers with CKD Co. Ltd. for their contribution in developing the special image processing hardware. References [l] K. Araki, Y. Sato, and S. Parthasarathy, "High Speed Rangefinder," SPIE, vol. 850, Optics, Illumination, and Image Sensing for Machine Vision, pp , 1987 [2] T. Kanade, A. Gruss, and L. Richard Carley, "A Very Fast VLSI Rangefinder," Proc. of 1991 IEEE Int. Conf. Robotics and Automation, pp , 1991 [3] Alexander, B.F. and Ng, K.C, "3-D Shape Measurement by Active Triangulation using an Array of Coded Light Stripes," SPIE Proceedings Vo1.850, Optics, Illumination, Image Sensing for Machine Vision 11, pp , 1987 [4] J. Peter Rosenfeld and Constantine J. Tsikos, "High-speed encoding projector for 3D imaging," SPIE, vo1.728, Optics, Illumination, and Image Sensing for Machine Vision, pp , 1986 [5] M. Matsuki and T. Ueda, " A Real-Time Sectional Image Measuring System Using Time Sequentially Coded Grating Method," IEEE Trans. PAMI, vol. 11, no. 11, pp , 1989 [6] K. Sat0 and S. Inokuchi, "Range-imaging system utilizing memetic liquid crystal mask," IEEE, 1st ICCV, pp , 1987 [7] J. L. Posdamer and M. D. Altschuler, "Surface Measurement by Space-encoded Projected Beam Systems," Computer Graphics and Image Processing, 18, 1982 [8] Y. Sato, K. Hattori, M. Otsuki, "Real-Time Handy Rangefinder Cubicscope," Proc. of 3rd Int. Conf. on Automation, Robotics, and Computer Vision (ICARCV '94), pp ,