STEREOSCOPIC ROBOT VISION SYSTEM

Transcription

1 Palinko Oskar, ipl. eng. Facult of Technical Sciences, Department of Inustrial Sstems Engineering an Management, Novi Sa, Dositej Obraovic Square 6, Serbia & Montenegro STEREOSCOPIC ROBOT VISION SYSTEM Abstract The visual sstem is one of the most important sensors in robotics. It is use b the robot to acquire information about the worl, to be able to navigate in it. A stereoscopic sstem allows the robot to easil etermine the istance of the objects in its vicinit. This paper iscusses a stereoscopic sstem, which allows the robot to locate a flight of stairs an to etermine its orientation. The main steps in solving this task are as follows: ege etection is applie on the stereo image; then the analtic form of theses eges are calculate using Hough transformation; stereo matching is one using the starting points of the eges; the istance of the stairs is erive using triangulation. Finall the orientation of the stairs is calculate using geometric equations. Kewors: machine vision, robotics 1. INTRODUCTIONAL REMARKS Robotics in the moern worl is gaining more an more significance. The fiel of robotics, which aims to create human-like robots is calle humanoi robotics. Robot vision is a ver broa scientific fiel. Some of its interests are: visual servoing, pattern recognition, stereo vision, etc. Stereo vision is one of the more important parts of robotics, because of its significance for moving in an unstructure environment. This paper introuces a stereo sstem esigne for use in anthropomorphic robots as for navigation in an unknown environment. The emphasis was set on etecting an analzing simple, prismatic objects like: stairs, holes, prismatic obstacles, etc. Special attention was evote to scenes containing stairs, which the robot analses an then approaches. A virtual simulation environment is programme for testing the esigne algorithms in ever phase of evelopment. Finall the sstem is teste on a small-scale mobile robot in a real-life human environment. 2. ELEMENTS OF MACHINE VISION This chapter introuces some of the basic elements of machine vision, which are of use later in this paper, like ege etection, feature etraction an triangulation. 2.1 Ege etection Ege etection is an image processing metho. At the places, where a function has an intense inclination, the first erivative will have a local etreme value [5]. In the case of 2D images, the mathematical operator graient is use. It is a two imensional erivative which is irecte towars the biggest rate of change in the vicinit of the point consiere. So as to etect eges, local etreme values of the graient must be foun. In the case of igital signals, the graient is approimate with the following equations: f (, ) f ( +, ) f (, ) = =, (1.1) f (, ) f (, + ) f (, ) = =, (1.2) where an is the horizontal an vertical istance between two ajacent samples (usuall equal to 1). The magnitue an orientation of the graient can be epresse as: M = +, = arctan. (1.3) The Cann metho This algorithm, in aition to the graient calculation, also contains other steps for improving the results of the etection. Its istinguishing marks are the two threshol values [6], which are to be eplaine in the following. The algorithm can be ivie into 6 steps: 1. In the first, a igital Gauss filter is applie to the image as to suppress the possible noise 2. After the elimination of noise, a 2D graient is use with etene convolution matrices:

2 3. The orientation of the graient is calculate using the formula (1.3). 4. In the fourth step the orientation of the graient of each point is classifie into one of 4 groups. For eample, lines with orientation between 0 an 22.5 egrees an the ones between an 180 are assigne into the class of 0. In this wa, the eact value of the orientation is substitute with classes. 5. Net, the non-maimum ege points are suppresse. In this step the ege line is followe b the class information. Ever ege point that is not in the orientation of the previous points class, gets eliminate. Onl those points are left which have corresponing orientation class. 6. In the last step, the continuit of the ege line is assure. Two threshols are introuce, p1 an p2, where p1>p2. All the points on the ege, which have intensit larger than p1 are automaticall confirme. If its intensit is less than p1 but more than p2 then the point is going to be confirme onl if it has a confirme point in an ajacent square. Otherwise it will be elete. This last step is the main innovation of the Cann metho, which makes it ver effective. In this paper, this metho will be use, because it gives narrow (5th step) an continuous (6th step) eges. secon space. If Hough transformation is applie on an image erive b ege etection (a binar image), then the brightest points in the ( r, ) plain will be the ones corresponing to the straight ege lines [4]. Fining local maimal points will iel in ege etecte line equations. The familiar form of an equation = a + b is gotten b epressing the parameters in the following wa: cos a =, sin 2.3 Stereoscop - triangulation r b =. (2.2) sin Stereoscop is a wa of seeing objects in 3D. The goal of it is to be able to etermine epth of view, object istance, object proportions, etc. in the scene. Triangulation is a wellknown metho of calculating the istance of objects knowing the angles uner which the object is seen from at least two positions an knowing the istance between those two positions [3]. In this wa triangulation is closel relate to stereoscop, because it gives information about istance, one of the most important elements of stereoscop. object 2.2 Hough transformation This transformation belongs to the group of feature etraction methos. It is intene to etract regular features [4] from the images, e.g. lines, circles, ellipses, etc. All of these forms can be epresse analticall (i.e. through an equation). In this paper the Hough transformation is use to etract analtic epressions of lines. Because of this, the methos subtpe esigne for lines will be eplaine in the following. A line can be parameterize in the - plane like: cos + sin = r, (2.1) where r is the length of the normal of the line, which normal intersects the coorinate origin (0,0), while is the angle between the normal an the ais. For an point on a particular line, the values of r an are constants. lens α projection plane Figure 2.2 Triangulation using two cameras Knowing the angles α an β as well as the istance z between the two cameras, the net equation can be erive to give the istance of the object: tgα tgβ = z. (2.3) tgα + tgβ z β r Figure 2.1 Parametric escription of a line In this wa, Hough transformation is a projection from the (, ) space into ( r, ) space. Equation (2.1) shows that points in the first space are reall sinusoial curves in the SIMULATION SYSTEM VIRTUAL SCENE The virtual scene is a 3D environment, which eists onl as a software simulation. It contains virtual objects, lights an virtual cameras. Cameras are use to get 2D images of the scene, just like in real worl. In the following, the main steps will be given to eplain how the simulation works. The first step is the acquisition of images from two stereo cameras, which are then processe an analze as get information on objects that are present in the scene. Uner analsis, we mean the etermination of istance of objects an their orientation compare to the camera. Finall the virtual robot approaches the object (stairs) so as to be parallel with the

3 front ege of it. It is important to emphasize, that the robot knows the geometr of the sstem onl through the camera images. 3.1 Virtual cameras Knowing the coorinates of a point on the camera image, it is eas to etermine the angle uner which it is seen b the camera: ( 2) tan α α = arctan ma, ma 2 where α ma is the with of the fiel of view, ma is the resolution of the camera. In this wa, the necessar angles are gotten for triangulation. The cameras are positione in a so-calle canonical configuration. That means that their optical aes are parallel, the projection surfaces are in the same plane an their upper eges belong to the same line. In this wa, the stereo pairing of a point on the stereo images is one on the same horizontal line. 3.3 Image processing The first step after acquisition is the transformation of color into intensit (grascale) images, because the following algorithms can work onl on such pictures. Then the Cann etection is invoke (escribe in 2.1.1). The binar pictures (black-white) of the etecte eges are sent to the blocks for Hough transformation, which etracts parametric information on the lines present in the scene (as escribe in 2.2). The result of this metho is a continuous 2D grascale image from which the local etremes must be etracte. As eplaine in 2.2 the maimum points represent the straight lines in the - plane. In this wa, the parametric line equations are gotten. 3.4 Analsis an reasoning The analsis begins with searching for the beginning an ening points of the parametric lines on the left-sie stereo image. It is one in the following wa: the analtic lines are followe until a iscontinuit in the ege is reache. If it is a iscontinuit from black to white ot, then the ege begins, otherwise it ens. These points are then stereo paire with the right-sie image. Pairing search is one on the same horizontal line as in which the point lies on the left image. Stereo pairing is a emaning process, that uses 2D correlation calculation for each point of the line [1]. The point with the best correlation result will be eclare as the right pair of the point on the left image. Correlation usuall gives goo results, because the to images are quite similar ue to the small parallaes. Calculating the angle The algorithm for calculating the angle entirel base on geometric equations. No approimations were use. Equation 2.2 shows how to calculate the angles in the fiel of view of the camera knowing its position on the camera image. This calculation is vali for both an -ais. In orer for the robot to be able to approach the stairs, it must know what is the angle between him an the object in the horizontal plane. This angle is esignate which must be foun knowing onl the angles in the image (), () an also the inclination an height of the camera. stepenište kamere Figure 3.1 Angles in a) horizontal an b) vertical plane After the euction of a series of geometric equations, we get: cos β = arctan tanα, (3.1) cosϕ where α is the horizontal angle in the picture, β is the vertical angle in the picture an ϕ is the camera inclination. The phenomenon of incline eges Using perspective projection, an interesting effect can be notice: near the borer of the image the lines that are in real-life parallel to the borer, appear to be incline. It oesn't happen with the lines going through the center of the image. The closer the line is to the borer, the phenomenon is more emphasize. Figure 3.2 The phenomenon of incline lines This effect appears when the lines are not parallel with the projection surface, because then some points on the line are situate closer to the surface than others. I.e. when the camera is incline forwar, the upper part of the projection plane is getting closer to the scene while the lower part is getting more istant. In that case when a ra of light comes from one of the upper corners, inclining the camera will cause the light to move up an awa from the center. This eviation can be correcte with the following equation: α = arctan. (3.2) 2 ma tan( ma 2) α β kamera ϕ projekciona površ 197

4 Calculating the istance of objects With the basic metho of triangulation the right-angle istance of the object is gotten. s e s δ λ e α M γ z Figure 3.3 Geometr in the triangulation plane But in this work the istance of the object from the central point M is neee: e =, (3.3) sin arctan z 2 tanφ where is gotten from equation (2.3) an the other elements are eplaine on figure 3.3. The calculate istance must be projecte on the - plane. Knowing the angle of inclination of the camera β, the task is trivial: e = esin β. (3.4) φ Figure 3.4 Calculation of the operational lengths The operational lengths s an calculate as follows from the figure: s e sin( + 90 λ) (3.6) = s e cos( + 90 λ) (3.7) = s for approach are Knowing them, the robot can perform its actions. These can be ivie into the following steps: a) b) Finall the projection of the angle γ on the - plane must be epresse: tanγ tan λ =. (3.5) sin β 3.5 Simulation of the robot's actions To perform the action of approaching the stairs the following information is neee: the angle in the horizontal plane between the robot an the object the projection (on the - plane) of the istance of the object from the central point, e c) o 90 ) s the angle λ It is enough to know onl these elements so as the robot can perform its actions in case a flight of stairs is in front of it. The robot must approach it so, that the front ege of the stairs must be parallel with the line connection the two cameras an ever time the robot must be on a constant istance. The starting position is in the general case as follows: o 90 s Figure 3.5 Robot actions o In step a) the robot turns in an angle of 90 so as the optical aes will be parallel to the front ege of the stairs. Then, in step b) the robot is moving straight forwar. The 198

5 istance it shoul prevail is equal to the sum of s an some value,. This value is ae so the robot oesn't approach eactl the left ege of the stairs, but somewhere in the mile. Step c) is turning back in a right angle. This wa, the robot is parallel to the front ege of the stairs. Finall the last step ) is performe in which the robot passes the straight istance of s. 4. EXPERIMENTAL VALIDATION The goal of eperimental realization of the visual sstem is to check the theoretical an simulation algorithm in real-life situation. The eperimental sstem consists of a mobile robot, that analzes the scene an then approaches the stairs. It is not a walking bipe (as the goal platform) but a wheele robot. The reason for this is of course that, the bipe is not prouce et, but the valiation must be performe. Figure 4.2 Camera image of the real-life stair moel stereo cameras notebook computer Figure 4.1 Mobile robot chassis The main parts of the robot are: the stereo cameras which are the most essential part of the sstem; two Logitech's QuickCam for Notebooks Pro web cameras were selecte for the task with a fiel of view of an a resolution of piels; the cameras must be mounte in the canonical wa a notebook computer which performs all the signal processing; an Acer Aspire 1312 is use with AMD Athlon GHz processor an with 256Mb of memor electronic circuits the are comprise of a AT89C52 controller (with its environment an RS232 communication with the PC) an a river electronics boar motors step motors are use for greater precision. The analsis of the scene is performe eactl like in the simulation. A test image an the results of the analsis are shown in the following: Figure 4.3 Detecte eges, Hough lines, istances, angles The eperimental results of etermining the istance of an object base on the above gotten pictures gave the following results: e racun [cm] e nom [cm] e [cm] δ e [%] 97,43 97,1 0,33 0,40 119,39 120,5 1,11 0,92 89,31 88,5 0,81 0,91 88,93 88,4 0,53 0,60 91,12 90, ,35 90,84 89, ,02 101,46 99,5 1,96 1,97 102,88 101,2 1,68 1,66 78, ,21 2,91 73,72 71,8 1,92 2,67 Table 4.1 Distance measurement results In further testing, eight cases of robot approach were conucte. The results were as follows: the robot succeee in si cases to precisel position itself in front of the stairs, while in two cases it in t. The errors were in tests number 4. an 6. Analsing the errors it mae, it was conclue, that in the first case the stereo matching sstem i not succee in its task ue to high noise in the images, while the other fault happene because an imperfection in the electro-mechanical part of the mobile robot. 199

6 5. CONCLUSION This paper emonstrate that the propose stereoscopic visual sstem is a viable solution of the require task. It was shown, that the problem of approaching a staircase can be solve solel with the use of the stereo images an knowing onl the camera inclination an height. The eucte geometrical equations in combination with some well-known image processing methos (like Cann ege etection, Hough transformation, etc.) are well suite for solving problems of this tpe. 6. DIRECTIONS OF NEXT RESEARCHES The presente visual sstem was projecte for work with prismatic objects. In further evelopment the eisting algorithm shoul be generalize to be able to work with objects that on t have straight eges. After than, when the robot woul be able to recognize a large range of ifferent obstacles, a cognitive sstem shoul be evelope using artificial intelligence. That woul mean that the robot woul be able to classif an learn to recognize new, unfamiliar tpes of objects. 7. REFERENCES [1] M. Sonka, V. Hlavac, R. Bole, Image Processing, Analsis an Machine Vision, Brooks an Cole Publishing, Iowa Cit, [2] D. Ballar, C. Brown, Computer Vision, Pretince-Hall Inc., Engelwoo Cliffs, [3] A. Marshall, Vision Sstems, last access: [4] R. Fisher, Hough Transform, last access: [5] B. Green, Ege Detection Tutorial, last access: [6] B. Green, Cann Ege Detection Tutorial, last access: