Obstacle Avoidance of Autonomous Mobile Robot using Stereo Vision Sensor

Transcription

1 Obstacle Avoidance of Autonomous Mobile Robot using Steeo Vision Senso Masako Kumano Akihisa Ohya Shin ichi Yuta Intelligent Robot Laboatoy Univesity of Tsukuba, Ibaaki, Japan {masako, ohya, Abstact The goal of this eseach is to develop a eal time obstacle avoidance system fo autonomous mobile obots using a steeo vision senso. At fist, an obstacle detection method is poposed. It is based on steeo measuement without any seach of the coesponding points to match them. This method is fast enough fo a mobile obot which has poo capabilities to cay out espectable image pocessing. Howeve thee happened a poblem that some ghost objects ae detected. We ll descibe the solution and an expeimental esult which shows the effectiveness of the impoved method. Then, a stategy of obstacle avoidance and an implementation of the poposed method to the mobile obot ae descibed. 1 Intoduction When mobile obots move in a eal envionment, ecognition of suounding objects is a big subject. Visual infomation is widely used fo navigation and obstacle detection of mobile obot[1-8]. In this eseach, we conside to use steeo vision senso. But steeo image pocessing method usually demands huge amount of time. The most impotant point fo eal-time sensing is how to deal with this infomation effectively. The pupose of this eseach is to ealize an autonomous obstacle avoidance in an usual envionment. To ealize that, both a eliable senso and a suitable path geneate system ae needed. In this pape, at fist we cove the obstacle detection method which pocessing is fast enough fo the obot to find 3D objects at quick ate. We also talk about detection failue with this method and how to impove it with an expeimental esult. Next, the system of whole motion contol contains oute unning and obstacle avoidance by planing suitable paths. Finally conclusion and futue woks ae descibed. Figue 1: Mobile obot equipped with steeo vision senso 2 Steeo Vision Senso The set of steeo vision sensos we use in this eseach is composed of two monochome CCD cameas equipped with about 9 degees wide-angle lenses, which ae fixed on the left and ight side with the same height at the top of the obot(see Figue 1). Two images ae captued synchonously on an image pocessing boad. It always equies lage mount of pocessing time to seach coesponding points between ight and left images fo ecognizing 3D objects. Some methods have been aleady poposed in ode to obtain data of obstacle existence at fast ate[9][1]. Thei pinciples ae as follows: Fist, all objects which ae taken in each image ae supposed to be completely dawn on the floo. Second, one image is estimated fom the othe by the matix calculated with elative position. Thid, compae each point on the eal image with the coesponding one on the estimated image. If thee is

2 steeo camea obstacle... coesponding points 25 2 left image ight image floo pape BRIGHTNESS[-255] RIGHT LEFT POINT NUMBER[-32] compae 25 Figue 2: The pinciple of obstacle detection in this eseach. It makes eal time obstacle detection possible without any seach of the coesponding points on each images.if both ight and left of the bightnesses of coesponding points ae almost equal, thee is not any obstacles thee. The diffeence of bightnesses means thee is something aound the point. can (obstacle) BRIGHTNESS[-255] RIGHT LEFT POINT NUMBER[-32] Figue 3: The sample esults of a flat(pape) and 3D(can: obstacle) objects. Hoizontal axis means coesponding points fom left to ight, vetical axis shows the bightness of each point, solid line is fo ight image and dotted one is fo left. Thee ae diffeences between ight and left value at the point aound both end of the can. a cetain diffeence of bightness, aound thee any 3D objects ae detected. These methods succeeded to shoten pocessing time. But most of them as usual need consideable time. Then we suggested an appopiate method fo obstacle detection as a eal time pocessing of the visual infomation(see Figue 2). In this method we also take advantage of the idea of the pevious method, but only fo a small aea in the images which is necessay to conside the futue motion of the obot. Instead of calculating the estimate image, we need to pepae the table of coesponding points in advance. Using it and just compaing each bightness value at the same point, we can undestand if thee is something at the supposed position o not. The detection needs only 35.2[msec], this numbe is an aveage of epeated 1 times detections accoding to the following conditions: Using a table which includes 15 coesponding points location data; they ae actually assumed to be on the floo, 4[cm], 65[cm] and 1[cm] in font of the obot in the line of 15[cm] length. Figue 4 shows those points location. 35.2[msec] includes 33[msec] of captue time, so it shows how fast this pocess is executed. An object of 5[cm] width 5[cm] height can be detected. y x 15cm 3cm 4cm 65cm 1cm Figue 4: The elative location of coesponding points fo steeo camea fom the obot.

3 bightness 25 2 (1) obstacle (doll) high contast (2) on the floo (3) eflected light left image coesponding point ight image 15 1 Aveage of 9 pixels (a) compae 5 bightness R bightness L (b) compae theshold bightness + edge + detected detection edges <1> [1] pixel summay pixel / 9 <2> [2] Figue 5: Expeimental esult dealing with obstacle(1), black tape on the floo(2) and eflected light(3), befoe (uppe pat) and afte (lowe pat) impovement. Hoizontal axis shows the location of the coesponding points on the floo. Uppe gaph s vetical axis displays bightnesses of those points and lowe gaph s one expesses the expeimental esult afte impovement. Aeas whee obstacles ae detected ae maked by coloed bas. In the uppe gaph, thee ae detection eos, in the lowe one they disappeaed thanks to impovement. 3 Impovement of Detection Method Howeve thee ae two detection eos that occu using this method. That means some ghost objects ae occasionally detected. One is caused by eflected lights (see the ight pictue of Figue 5), and the othe is by high contast on the floo (the middle one). 3.1 Reflected ceiling lights on the floo Fist eo is caused by eflected ceiling lights on the floo. The cause is the position whee lights ae eflected in the image depends on the location of the ight and left cameas, it leads to a cetain diffeence of bightness at the same coesponding points. It can seen at Figue 5 ight pixel Figue 6: Impoved method fo high contast on the floo hand uppe gaph(3), that thee is nothing on the floo but senso has made wong judgment.( Coloed bas mean that obstacles ae existing aound them.) Howeve, thee is a big diffeence between a eflected light and a eal object, that is how distinct thei edges ae. Reflection of light is blued, wheeas a eal object has a clea contast fom its suoundings. So it is possible to distinguish eal objects fom eflections. Lowe gaph of Figue 5 has two types of lines. Result of this impovement fo eflected lights is showed at solid line 1. It pesents the value of each point that is affected by both its bightness and edge intensity calculated by diffeence of the point s bightness and next point s one. The shape the edge of objects aound the point, the lage the value of the point becomes. Cetain theshold has been set and some expeiments have been made. The detection eos elated to eflected lights have been canceled. Figue 5 s ight hand gaphs show that this method woks successfully. 3.2 High contast on the floo Second eo happens aound the bode on the floo between diffeent colos which intensities ae emakably changed(see Figue 5 uppe (2)). The eason we guess is the coesponding points on each ight and left image which ae peviously calibated using oiginal pocess ae not coesponding stictly. But exactness of calibation has a limit, and it is necessay to impove the object detection method with allowable magins of exactness. Then we eplace the detection algoithm by a moe toleant one. Pevious method compaes the aveages of nine points bightnesses shown in Figue 6(a). New one compaes the bightness of the points shown in Figue 6(b). If a cetain numbe of coesponding points have diffeent bightness, we deduce the existence of an obstacle aound these points. So dotted line 2 in Figue 5 lowe gaph,

4 coesponds to integes fom to 9. Hee theshold is set to 4. Afte this impovement, eos ae significantly deceased. When both 1 and 2 ae geate than cetain thesholds, eal obstacle exists aound the point. The judgment if thee is an obstacle o not is the esult of the logical and between the two compaisons pesented above. Thus expeimental esults show that both implementations wee successful. This can be seen on Figue 5 whee the coloed bas disappeaed in the case of (2) and (3) afte the new pocessing wheeas they emained pesent in the case of a eal obstacle. 4 Realization of Obstacle Avoidance Behavio Fist we decided some peconditions fo autonomous obstacle avoidance by mobile obot. Robot aleady knows the path to follow which is given by human, uses the infomation of path and obstacle existence to plan a suitable new path to the goal by itself. Theefoe the oute desciption affects evey decision, and in the beginning we need to define its details. What kind of motions ae suitable fo ou obot equipped with steeo vision senso on the top of it? Mention senso infomation, its sensing aea depends on the motion of obot itself and the aea has limit, that means to ealize totally safe movement equies to plan the paths only in wellknown envionment. Convesely some motions contain unning on unknown aea, theefoe spin tun should be banned. Fo these easons, each unning path is descibed as a staight line o an ac. Whole oute desciption which binds each paths smoothly becomes a taget oute to follow. Next discussion is about how to deal with those paths conveniently. We adopted a state tansition list, egading a staight line o an ac as a statement. One state contains its state numbe, which motion (line o ac), stat and end positions x y θ, the adius of the cicle (if state of ac) and the numbe of next state(shown at Table 1). Using the above list and the pesevation of cuent state, the obot can contol its objective behavio. While the obot executes geneal motion the list is handled in ode. If any obstacles ae found, suitable avoidance and ecovey ae demanded. So we conside that the list dynamically changes accoding to the detected point and can be pefomed suitably in ode to accomplish both avoidance and etuning. Concete means ae the following: 1) Plan new oute which avoids the point whee an obstacle has been detected. 2) Afte state pesevation pat ecognizes that a new state was added at the end of the list, Table 1: Oiginal state tansition list type of oigin point temination next adius oute x y θ x y θ state staight x b y b θ b x e y e θ e 1 1 ac x b1 y b1 θ b1 1 x e1 y e1 θ e1 2 2 ac x b2 y b2 θ b2 2 x e2 y e2 θ e2 3 3 staight x b3 y b3 θ b3 x e3 y e3 θ e3 4 C C taget oute Figue 7: The method to constuct avoidance paths. The position whee an obstacle is detected (O), the cuent position of the obot (A) and the oiginal oute list (Table 1) ae used. [B has to be calculated as the point whee a obot ecove to the oiginal oute, C is decided as a elay point between A and B. The paths ae detemined by those thee points using some ac lines.] B d A A O

5 Table 2: The list afte geneation of avoidance paths type of oigin point temination next adius oute x y θ x y θ state staight x b y b θ b x e y e θ e 1 1 ac x b1 y b1 θ b1 1 x e1 y e1 θ e ac x b2 y b2 θ b2 2 x e2 y e2 θ e2 3 3 staight x b3 y b3 θ b3 x e3 y e3 θ e3 4 : : : : 22 ac x b22 y b22 θ b22 22 x e22 y e22 θ e ac x b23 y b23 θ b23 23 x e23 y e23 yθ e23 3 it moves the pointe of cuent state to top of new state. 3) The last state s next state will be changed as the state which contains the ecoveing point. Using the list like this, just updating it enables the management motion of the obot as well as we expected. Image Input CCD camea Extenal Synchonized Signal Input Image Pocessing Maste Contol Locomotion "!$#&%('*) Coespondence between each Modules CCD camea Figue 8: Steeo vision senso system 5 Planing Avoidance Paths When the obstacle is found, the path should be bended by eplacing the oiginal path with a new path to avoid it. The new path can be constucted using the detected obstacle position and cuent obot position(see Figue 7). Next the points whee obot should pass though to avoid an obstacle and ecove to oiginal oute ae demanded to be decided. The ecovey point is defined as follows: Calculate two points whee the line accoding to the given list, and the cicle its cente is detected obstacle position and its adius is the length between detected obstacle position and cuent obot position coss each othe. The point which does not coespond to the cuent obot position is defined as the ecovey position to the oiginal oute. Also the position to pass is the position whee both half of the angle made by above mentioned thee points and fa cetain distance fom the detected position. Consideing the size of obot, the distance is befoehand decided. The same conditions mentioned in last section ae also adopted to new oute as avoidance ones, so evey path should be descibed by a staight line o an ac. Hee evey path can be geneated as two acs when the location and oientation of two end points ae given. Fo smooth motion of the obot the adius of each ac would like to be as lage as possible. That means above two acs designed the same timing will become equal adius. Following those ules, a unique avoidance path can be constucted fom the infomation of the detected obstacle position, cuent obot position and the oiginal oute list. 6 Implementation In this eseach we use the mobile obot YAMABICO [11] with a steeo vision senso (Figue 1). Maste contol, locomotion contol and image pocessing boads put on the ack of obot have each own CPU (Tanspute T85 /2MHz) seen at Figue 8. We can contol its motion easily thanks to oiginal softwaes that have aleady been developed in ou laboatoy. A high level command, fo example, Tack the staight line which includes position(x,y) and its oientation is theta on the pedefined coodinates system,can contol the obot exactly. The system we implemented is composed as follows: The above mentioned state tansition list, the cuent state management pocess, the oute unne pocess which contol the obot s motion accoding to the list, the pocess geneating new paths using the detection infomation and cuent obot position. Those wok each othe to aim at complete expected motions including both oute unning and obstacle avoidance. 7 Conclusion In this pape we descibed high speed obstacle detection method and the way to decease its detection eos. Then we pesented the method of obstacle avoidance and its implementation. It aleady confimed that obot autonomously uns accoding to given list, it poves that some of necessay functions wee successfully implemented. The est of implementation is the path geneation

6 position estimate obot s cuent position path geneato fo obstacle avoidance Locomotion Module moto contol contol command oute unne oute list pediction of uncetainties, CVGIP Image Undestanding, vol. 56, no. 3, pp , Nov [3] K. Sugihaa, Some location poblems fo obot navigation using a single camea, Compute Vision, Gaphics and Image Pocessing, vol. 42, no. 1, pp , Ap [4] T. Tsubouchi and S. Yuta, Map assisted vision system of mobile obots fo eckoning in a building envionment, in Poc. IEEE Int. Conf. Robotics Automat., Ma./Ap. 1987, pp detection data obstacle detection Maste Contol boad Vision Pocessing boad pocess Figue 9: The sketch of motion contol pocesses data flow pocesses. Futue task is to complete whole implementation and conduct some expeiences in vaious envionment with seveal obstacles to evaluate this system. Ou final goal is eal-time autonomous execution of given movement avoiding obstacles natually. Acknowledgement The authos would like to thank M. Launay Fabien fo the caeful poofeading fo pepaing this manuscipt. We also thank vey much M. Tomoaki Yoshida of the Univesity of Tsukuba fo his useful suggestions and suppot of this eseach. Refeences [1] T. Camus, D. Coombs, M. Heman and T. Hong, Real-time single-wokstation obstacle avoidance using only wide-field flow divegence, in Poc. 13th Int. Conf. Patten Recognition, Aug [5] Y. Matsumoto, M. Inaba and H. Inoue, Visual navigation using view-sequenced oute epesentation, in Poc. IEEE Int. Conf. Robotics Automat., Ap. 1996, pp [6] T. Ohno, A. Ohya and S. Yuta, Autonomous navigation fo mobile obots efeing pe-ecoded image sequence, in Poc. IEEE/RSJ Int. Conf. Intelligent Robots Syst., Nov. 1996, pp [7] A. J. Muñoz and J. González, Localizing mobile obots with a single camea in stuctued envionments, in Poc Wold Automation Congess Robotic and Manufactuing Systems, May. 1996, pp [8] D. J. Kiegman, E. Tiendl and T. O. Binfod, Steeo vision and navigation in buildings fo mobile obots, IEEE Tans. Robotics Automat., vol. 5, no. 6, pp , Dec [9] Guo-Wei Zhao, Shin ichi Yuta: "Obstacle Detection by Vision System fo an Autonomous Vehicle", Intelligent Vehicles Symposium, pp.31-36, 1993 [1] K. Onoguchi, N. Takeda and M. Watanabe: "Plana Pojection Steeopsis Method fo Road Extaction", Intenational Confeence on Intelligent Robots and Systems, vol.1 pp , 1995 [11] S. Yuta, S. Suzuki and S. Iida, Implementation of a small size expeimental self-contained autonomous obot sensos, vehicle contol, and desciption of senso based on behavio, R. Chatila et al. Eds, Expeimental Robotics II, Spinge-Velag, 1993, pp [2] A. Kosaka and A. Kak, Fast vision-guided mobile obot navigation using model-based easoning and