Abstract. 1. Introduction. IEEE International Conference on Emerging Technologies September 17-18, Islamabad

Transcription

1 Abdul Bais, Robert Sablatnig and Gregor Novak Institute of Computer Technology Vienna University of Technology, Vienna, Austria Pattern Recognition and Image Processing Group Institute of Computer Aided Automation Vienna University of Technology, Vienna, Austria {bais, Abstract This paper 1 describes visual landmark extraction for self-localization of an autonomous mobile robot in a wellknown dynamic environment. The gradient based Hough transform provides the strongest groupings of collinear pixels having roughly the same edge orientation. Groups of pixels are then processed to calculate length and end points of line segments, which together with the length and direction of the normal completes the description of the line segments. This is followed by classification of field markings which are bright lines and arcs on a dark background forming double edges of opposite gradient. Corners, junctions and line intersections are determined by the interpretation of detected line segments. Simulation results illustrate the performance of the method using both real and synthetic images. 1. Introduction In mobile robotics, the basic requirement for autonomous navigation in any environment is self-localization i.e. determining the robot s own position and orientation. There are two different approaches to solve this problem: global position estimation and local position tracking. In global position estimation the robot is only given a map of the environment, while in local position tracking it also has an estimate of its initial position. Methods for local position tracking are very efficient and provide good short term position estimates but suffer from unbounded error growth due to integration of minute measurements to obtain the final estimate. Whereas, techniques for global position estimation, are less accurate and often require significantly more computational power [1]. This leads to new techniques [2 10] where local measurements are fused with measurements from the robot en- Supported by Higher Education Commission of Pakistan. 1 This paper was written within the Center of Excellence for Autonomous Systems of the Vienna University of Technology (CEAS). vironment in order to mitigate the unbounded growth of errors. However, the robot must be able to estimate its position from scratch as it may not be possible or at least desirable to give the robot with an initial estimate of its position or the robot may lose track of its position during navigation. In this paper we focus on visual landmark extraction for the self-localization of a tiny mobile robot in a well known dynamic environment consisting of landmarks i.e. lines, corners, junctions and line intersections. The test bed for our algorithm is a soccer playing robot called Tinyphoon ( [11]. Tinyphoon is a two wheeled robot equipped with stereo cameras on a pivoted head. Currently, soccer robots of the size of Tinyphoon are marked on their top with some color patterns, which are then tracked for position estimation using a global camera and a host computer. We aim at a shift towards complete autonomy, where all sensing and processing will be onboard. However, we look at localization techniques of other (much bigger and slower) soccer robots and also of indoor robots with techniques that could be applied to our domain. Gutmann and Schlegel [12] use dense range scans of the surrounding walls for localization of their robots. Iocchi and Nardi [2] use the Hough transform [13] to extract line segments from range data. The main drawback of a range sensor is that it requires the environment to be surrounded by walls. In more natural wall free field, vision will be the only effective sensor. However, calculating range of each image pixel using stereo cameras would be too expensive computationally. Vision based approaches reported in [3, 4] are based on detecting color marked features such as goal posts and corners. In such context, localization depends highly on robust recognition of color markings that have to be explored around the field. For robots with common cameras, searching these marks distracts them from concentrating on game objects [9]. Adorni et al. [6] use two wide angle cameras for local /05/$ IEEE 132

2 ization of their goal keeper robot. The robot uses the white lines of the penalty area as landmarks which are detected by applying the Hough transform to those image pixels that are segmented as white after a color segmentation and belong to an edge. The very specific details of the method make it difficult to be adopted by other robots and furthermore it is hard for the same robot to localize itself in other parts of the field. Utz et al. [8] detect image pixels belonging to field markings on the basis of color transition, which are then transformed into the Hough parameter space. Filtering feature points that may belong to line segments on the basis of color transitions makes the method highly dependent on color segmentation. Huebner [9] detect pixels belonging to field lines based on their symmetry in the image. These pixels are then grouped into line segments using local operations. This method can only detect field markings. Merging of long occluded lines will be a problem. Herrero-Pérez et al. [10] detect corner features made by field lines. Corners are detected using a method from Sojka [14] which is based on the variance of the gradient of the brightness. Corners produced by this method are then filtered if they are not coming from white line segments. Christensen et al. [7] use the current position estimate (from odometry) of the robot to make projections from the 3D CAD model of the environment corresponding to a prediction of what a camera is expected to see from a given view-point. These projections are then used for matching against line segments extracted from images. Such method will only work if the current position estimate is close to the actual one. The main difference between this current method and the previous methods is that we detect line segments, segment them if they belong to field markings, detect corners, junctions and line intersections using semantic interpretations of these segments. These features are then input to a stereo algorithm that will compute 3D position, which is finally used in the computation of the robot position and orientation. The ballance of the paper is organized as follows: Section 2 discusses extraction of line segment extraction based on the Hough transform. These line segments are then tested if they are field markings in Section 3 followed by detection of corners, junctions and line intersection in Section 4. Experimental results are presented in Section 5, finally the paper is concluded in Section Detecting line features Our primary target is to extract the global structure of line segments, in the presence of heavy occlusions. Different local line detection schemes [15 17] together with variants of the Hough transform (see [18] and [19]) were reviewed. Local methods were dropped as they could not handle a very high level of occlusion and have very complex implementation details. The probabilistic Hough transform methods are the most recent developments and aim at achieving high computational speeds but they require some stopping criteria Hough transform The Hough transform is an effective and robust method for finding straight lines fitting 2D points in images. Duda and Hart [13] suggested a parametrization of the straight line ( 1), which says that any point (x,y) can be represented by the length (ρ) and angle (θ) of the normal vector to the line passing through this point from the center of the coordinate system (see Fig. 1). ρ(θ) = x cos θ + y sin θ (1) Figure 1. Line parametrization with ρ and θ According to ( 1) each point (x, y) in the image corresponds to a curve, in the parameter space. Curves corresponding to colinear points of a line with parameters (ρ 1, θ 1 ) will cross each other at point (ρ 1, θ 1 ) in the parameter space. Both θ and ρ are continuous variables which are quantized with step sizes δθ and δρ, respectively. This quantization results in a two-dimensional accumulator array, A. From the global perspective, an image pixel could be part of infinite number of lines which means that ( 1) will be calculated for all values of θ ranging from 0 to π (excluding π). However, as image points that are supposed to be transformed into the parameter space are the output of an edge detector, which also gives orientation of the edge passing though that point. Hence, it is possible to calculate ρ(θ) for those values of θ which are close to the direction of edge normal, φ. This closeness could be determined by a given tolerance Peak detection Peak detection could be seen as the inverse of the Hough transform as peaks in the accumulator array represent colinear points in the image. These peaks may belong to the 133

3 strongest set of lines. Peak detection is accomplished by searching through the accumulator array and choosing cells that meet certain criteria. We have implemented an iterative peak selection process [20]. This is a simple way of peak detection and better results could be achieved by filtering (e.g. the butterfly filter [21]) the parameter space before searching for peaks. The selected peaks may be spurious. The next section discusses verification of these peaks and also completes the line description. The parameters (θ, ρ), length l and the coordinates of the end points (p 1, p 2 ) constitute the complete line segment description [22] Peak verification and completing line segment description Input to this process is the list of all feature points and a list containing the detected peaks (parameter list). The parameter list is sorted in descending order. This process is outlined as follows. 1. Find indices of pixels that might have voted for this peak. 2. Test if edge orientation of each pixel is in compliance with θ. 3. For each pixel in the list add its 8-connected neighbor to the list if it satisfies the criteria for the edge normal. Sort the list. 4. Rotate the pixels such that they lie along the x-axis [20] (see Fig. 2). 5. Find gaps (if any) greater than Gap min. 6. Find length and end points of the segments separated by Gap min. Accept this segment if its length is greater than L min. 7. Merge two segments if the separation between their end points is less than Gap max. 8. Remove these pixels from the main list of feature points. In step 8, removing pixels from the main list of feature points helps accelerate the process and reduces the possible interference that they may cause in the verification of other line segments. This process is repeated for all elements of the parameter list. This process of line verification and calculation of additional parameters is iterative and is computationally expensive. There are other state of the art non-iterative approaches that are based on the fact that the spread of votes Figure 2. Rotating line segments in the accumulator array is dependent on the length and position of line segments. Atiquzzaman and Akhtar [22] calculate the complete description of single line segment in an image. They select some arbitrary column θ k in the accumulator array at a known distance θ p θ k from the detected peak. This column is then scanned for the first and last non-zero accumulator cells on which the calculation of end points of the line segment is based. This method can successfully calculate the end points and consequently the length and normal if there is only one line segment in the image. A more detailed analysis of the spreading of votes around a peak (formation of the butterfly) is reported by Kamat-Sadekar and Ganesan [23]. This method is an extension of [22] to determine a complete description of multiple line segments. Due to their non-iterative nature, these methods are very efficient. However, real world images are too complex and these butterf lies around the detected peaks will be heavily distorted. 3. Detecting field markings In this section we discuss the classification of line segments. All line segments extracted are tested if they are field markings or belong to some other objects. Field markings are white lines/arcs drawn on a dark background. In terms of gray scale gradient, such markings can be seen as a sequence of a negative gradient followed by an opposite positive gradient of equal magnitude or vice versa as shown in Fig. 3 (in Fig. 3 negative values are shown black). This verification is done between step 3 and step 4 of the algorithm discussed in Sub Section 2.3. The algorithm is as follows (see Fig. 4). 1. For an element (x, y) of the filtered list, start a loop (depending on line orientation) with counter x or y, that runs from 1 to W. 2. Calculate y or x, as y = round(x tan θ) or x = round( y tan θ ), respectively 3. Test if (x + x, y + y ) is an edge element and the absolute difference in orientation between (x, y) and (x + x, y + y ) is roughly

4 Figure 3. Dual edges of field markings (c) (d) Figure 5. Finding line intersection Figure 4. Finding field markings 4. If the test in the previous step fails then repeat Step 3 for (x x, y y ). 5. If either of the previous tests is successful then segment (x, y) as a field marking and terminate the process 6. Start the next iteration of the loop if the end has not yet reached. The value of W depends on the maximum allowed width of a field marking in terms of image pixels. As we are looking for nearly parallel line segments we move perpendicular to the line in both the directions. The verification of a few pixels in general suffices to decide whether the line segment is a field marking or not. 4. Detecting corners and junctions The completely described lines segments are used to detect junctions (T-junction and Y-junction), corners and line intersections. n line segments in an image may result in n C 2 = n(n 1) 2 intersection points. These intersection points can be calculated by solving ( 1) for all combinations of two line segments. The intersection point may not lie close to one or both of the line segments (Fig. 5(d)). The process of calculating the intersection point and its verification for a pair of line segments l 1 and l 2 is given as follows. 1. Calculate the intersection point (p) and test if it lie inside the image. 2. Rotate end points of l 1 and p (Fig. 5). 3. Test if the rotated x-component of the intersection point lies in between the rotated x-component of the end points or close to one of them 4. Repeat step 3 and 4 for l 2 This closeness (in step 3) is determined with the help of a threshold which is dependent on the amount of distortion that can be tolerated in the position of an intersection. The position of an intersection point with respect to the end points of the line segments helps to classify an intersection as a corner or a junction of a given type. Furthermore, the information about the type of line segments (as a field marking or not) helps to recognize intersection points uniquely. In case there are m (with m > 2) line segments meeting at one point there are m C 2 = m(m 1) 2 intersection points close to one another. We take the average as the required position of the junction if two or more intersections are within a given tolerance. The information about the end points of line segments allows to detect corners and junctions with this simple method. On the average we have line segments which makes this method very attractive. Davies [24] introduced an approach to corner detection based on the generalized Hough transform [25]. This method can be used to find corners which may be blunt or occluded. Barret and Peterson [26] detect corners, junctions and line intersection by exploiting the accumulator array of the Hough transform. After the generation of the parameter space and detection of peaks they perform a second pass through the edge map and compute the line integral over each sinusoid that corresponds to the current edge point. This method could also be applied to detect a virtual junction if all image pixels are considered as edge pixels in the second pass. 135

5 Figure 6. Testing with real images Table 1. Real World Image Total Detected Missed Spurious Lines Field markings Corners Junctions Line intersection Table 2. Sythetic Images Total Detected Missed Spurious Lines Field markings Corners Junctions els, respectively. Whereas, the tolerance for corners and junctions to be selected was 15 pixels of the rotated lines. 6. Conclusion (c) (d) Figure 7. Synthetic images Another approach is that of detecting the corners directly by template matching. But, such methods are computationally expensive. For example Q M N 2 n 2 operations have to be performed to detect corners using M n n pixel templates corresponding to M possible orientations of a corner with Q different angles in an N N pixel image [24]. 5. Experimental results We investigate the performance of the algorithm on synthetic as well as real images. Fig. 6 shows the edge map (with detected line segments and corners superimposed) on the real world image of Fig. 6. Fig. 7 shows the results of applying the algorithm to synthetic images. These images are generated from the 3D environment model of the FIRA MiroSot Small League( The corners are blunt as isosceles triangles are placed in the corners so that the ball is not trapped. The results for the real image and synthetics images are shown in Table 1 and Table 2, respectively. All images are pixels, the quantization steps for θ and ρ were 1 and 3 pixels, respectively. The tolerance used for the gradient based Hough transform is ±15. L min, Gap min and Gap max were set to 10, 10 and 50 pix- The method presented in this paper successfully extracts globally significant line segments from camera images. The global nature of the Hough transform extracts the strongest groups of collinear pixels. Within each group, spatially separated (sub-groups of) pixels are merged if they are locally significant. The parameters Gap min, Gap max and L min make the merging process robust against random noise. We calculate the intersection point of two line segments based on ρ and θ values of selected peaks. Further improvements in performance could be achieved if these values are recalculated once the line segments are extracted [22]. References [1] J. Borenstein, H. R. Everett, and L. Feng, Navigating Mobile Robots: Systems and Techniques. A. K. Peters, Ltd., [2] L.Iocchi and D. Nardi, Hough localization for mobile robots in polygonal environments, Robotics and Autonomous Systems, vol. 40, no. 1, pp , July [3] S. Enderle, M. Ritter, D. Fox, S. Sablatnög, G. Kraetzschmar, and G. Palm, Soccer robot localization using sporadic visual features, in International Conference on Intelligent Autonomous Systems 6 (IAS-6), E. P. et al., Ed. Amsterdam, The Netherlands: IOS Press, 2000, pp [4] A. Motomura, T. Matsuoka, and T. Hasegawa, Selflocalization method using two landmarks and dead reckoning for autonomous mobile soccer robots, in 136

6 RoboCup 2003: Robot Soccer World Cup VII, ser. LNCS, 2003, pp [5] F. de Jong, J. Caarls, R. Bartelds, and P. Jonker, A two-tiered approach to self-localization, in RoboCup 2001: Robot Soccer World Cup V. Springer-Verlag, 2002, pp [6] G. Adorni, S. Cagnoni, and M. Mordonini, Landmark-based robot self-localization: a case study for the robocup goal-keeper, in Proceedings of the International Conference on Information Intelligence and Systems, Bethesda, MD, USA, October 1999, pp [7] H. I. Christensen, N. O. Kirkeby, S. Kristensen, and L. Knudsen, Model-driven vision for in-door navigation, Robotics and Autonomous Systems, vol. 12, no. 3-4, pp , April [8] H. Utz, A. Neubeck, G. Mayer, and G. Kraetzschmar, Improving vision-based self-localization, in RoboCup-VI, ser. LNCS, G. K. et al., Ed., no Springer-Verlag, 2002, pp [9] K. Huebner, A symmetry operator and its application to the robocup, in RoboCup 2003: Robot Soccer World Cup VII, ser. LNCS, D. P. et al., Ed., vol. 3020, August 2004, pp [10] D. Herrero-Pérez, H. Martínez-Barberá, and A. Saffiotti, Fuzzy self-localization using natural features in the four-legged league, in RoboCup 2004: Robot Soccer World Cup VIII, ser. LNCS, D. N. et al., Ed. Springer-Verlag, 2005, pp [11] G. Novak and S. Mahlknecht, TINYPHOON a tiny autonomous mobile robot, in IEEE International Symposium on Industrial Electronics (ISIE 05), June 2005, pp [12] J. Gutmann and C. Schlegel, Amos: Comparison of scan matching approaches for self-localization in indoor environments, in 1st Euro micro Workshop on Advanced Mobile Robots. IEEE Computer Society Press, [13] R. Duda and P. Hart, Use of the Hough transformation to detect lines and curves in the pictures, Communications of the ACM, vol. 15, no. 1, pp , [14] E. Sojka, A new and efficient algorithm for detecting the corners in digital images, in Proceedings of the 24th DAGM Symposium, ser. LNCS, L. V. Gool, Ed., vol Springer, 2002, pp [15] J. B. Burns, A. R. Hanson, and E. M. Riseman, Extracting straight lines, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, no. 4, pp , [16] D. S. Guru, B. H. Shekar, and P. Nagabhushan, A simple and robust line detection algorithm based on small eigenvalue analysis, Pattern Recogn. Lett., vol. 25, no. 1, pp. 1 13, [17] S. Climer and S. K. Bhatia, Local lines: A linear time line detector, Pattern Recognition Letters, vol. 24, pp , [18] V. Leavers, Survey - which hough transform? Computer Vision, Graphics, and Image Processing: Image Understanding, vol. 58, pp , [19] H. Kälviäinen, P. Hirvonen, L. Xu, and E. Oja, Probabilistic and non-probabilistic hough transforms: Overview and comparisons, Image and Vision Computing, vol. 13, no. 4, pp , May [20] R. C. Gonzalez, R. E. Woods, and S. L. Eddins, Digital Image Processing using MATLAB, 1st ed. Prentice Hall, September [21] V. Leavers and J. Boyce, The radon transform and its application to shape parameterization in machine vision, Image and Vision Computing, vol. 5, no. 2, pp , May [22] M. Atiquzzaman and M. W. Akhtar, Complete line segment description using the hough transform, Image and Vision Computing, vol. 12, no. 5, pp , June [23] V. Kamat-Sadekar and S. Ganesan, Complete description of multiple line segments using the hough transform, Image and Vision Computing, vol. 16, no. 9-10, pp , July [24] E.R.Davies, Application of the generalised hough transform to corner detection, Computers and Digital Techniques, IEE Proceedings-E, vol. 135, no. 1, pp , January [25] D. H. Ballard, Generalizing the hough transform to detect arbitrary shapes, Pattern Recognition, vol. 13, no. 2, pp , [26] W. A. Barrett and K. D. Petersen, Houghing the hough: Peak collection for detection of corners, junctions and line intersections, in Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR 01), vol. 2, December 2001, pp