Hough Transform Run Length Encoding for Real-Time Image Processing

Transcription

1 IMTC 25 Instrumentation and Measurement Technology Conference Ottawa, Canada, May 25 Hough Transform Run Length Encoding for Real-Time Image Processing C. H. Messom 1, G. Sen Gupta 2,3, S. Demidenko 4 1 IIMS, Massey University, Albany, New Zealand 2 IIS&T, Massey University, Palmerston North, New Zealand 3 School of EEE, Singapore Polytechnic, 5 Dover Road, Singapore 4 School of Engineering & Science, Monash University, Kuala Lumpur, Malaysia C.H.Messom@massey.ac.nz, G.SenGupta@massey.ac.nz, Serge.Demidenko@engsci.monash.edu.my Abstract This paper introduces a real-time image processing algorithm based on run length encoding (RLE) for a vision based intelligent controller of a Humanoid Robot system. The RLE algorithms identify objects in the image providing their size and position. A RLE Hough transform is also presented for recognition of landmarks in the image to aid robot localization. The vision system presented has been tested by simulating the dynamics of the robot system as well as the image processing subsystem. The real-time image processing and control algorithms allow the unstable dynamic model of the biped robot to be controlled Keywords Run Length Encoding, Hough Transform, Real-time Image processing, Edge detection I. INTRODUCTION A vision based humanoid robot system requires a highspeed vision system that does not introduce significant delays in the control loop. This paper presents a vision system for biped control that performs in real-time. The humanoid robot used to test the system is a 12-degree of freedom biped robot. The image from the camera attached to the top of the robot is processed to identify positions of obstacles as well as any landmarks in the field of view. Obstacles can be accurately placed relative to the robot, and with the identification of land marks the robot can be accurately localized and a map of the obstacles developed in world coordinates. Once the objects have been located in the 2 dimensional image, a coordinate transformation based on the fact that the ground is level and all the joint angles are available, allows us to determine the object s position relative to the camera. If the joint angles are not available, an approximation of the camera position and orientation must be calculated based on the image from the camera. Visual features that will contribute to this calculation include position of the horizon and any gravitationally vertical lines in the image. The first part of this paper discusses the run length encoding algorithm (RLE) [1-3] applied to a simulated biped robot system. This algorithm allows objects to be identified in the field of view based on color and size. The second part of this paper discusses the localization problem based on landmark identification using edge detection and RLE. Given large landmarks such as the horizon or large obstacles filtering short lines effectively removes noise due to multiple small objects in the field of view. One disadvantage with the edge detection algorithms is the computational time associated with detecting and processing the edge image. This paper introduces a RLE edge representation to improve the performance of edge processing. II. BACKGROUND Much early work in biped and humanoid robotics has focused on the basic dynamics and control of the biped robot system [4-7]. However more recently researchers have started to address the higher-level functionality such as biped robot vision for navigation and localization. To test and develop this functionality toolkits that support full simulation of the vision and control system have been developed [8]. This study builds upon the vision enabled robot simulation environment using the 12-degree of freedom m2 biped robot [5, 6, 9]. A typical view from the robot in an environment with obstacles is shown in figure 1. The key objective of the vision system is to identify the objects in view given changing viewing angles and lighting conditions. This requires object s characteristic color and size to be continuously updated based on current conditions. Fig. 1. Biped Robot View

2 Image Capture RLE image processing Identified objects Hough Transform based object recognition Fig. 2. Image Processing pipeline Recently researchers have investigated biped vision strategies based on both simulation [1, 11] and real robot systems [1 13]. Braunl reported some of the problems associated with a reality gap when transferring results from a simulated system to a real robot system. Ensuring that the systems developed in simulation are not dependent on specifics of the simulation system ensure that this reality gap can be closed to the point that simulated solutions are useful for solving the real problem. III. IMAGE PROCESSING PIPELINE Figure 2 shows the image processing pipeline for the system presented in this paper. The core image processing algorithm used is the RLE based image segmentation and object tracking. This subsystem [3] provides a real-time object tracking algorithm. Its weakness is that it needs the range of color space occupied by the objects being tracked to be specified. This requires an environment with uniform lighting and little variation over time in order for robust performance to be achieved. To use the RLE algorithm in an environment with unknown objects and varying light conditions the required color space range values must be dynamically updated. This study uses a Hough Transform based edge detection technique [14] to identify new objects and their associated color space range values. This phase of the processing is slow and so runs in a separate low priority thread concurrent to the real-time RLE image processing. S v = T, S h = S v (1) Fig. 3. Edge detected image IV. EDGE DETECTION Edge detection is often used to identify objects and regions of interest in an image where there can be significant variation in size and colors of the objects of interest or the colors of the objects of interest are not known. In this paper edge detection is used to identify landmarks in the image, particularly the horizon and any unknown large obstacles. A 5x5 RGB Sobel edge detection filter (S v and S h see equation 1) can be applied to the raw RGB image (figure 1) producing the edge detected image (figure 3). This edge detection technique is computationally relatively expensive as compared to RLE, however in the situation where color identifiers are unknown it provides a suitable image that can be further processed to find information about the environment in which the robot is operating. In the simulated domain studied one of the key features that can be identified from the edge-detected image are the horizon from which the body position of the robot can be inferred (this is useful if the joint angles are not explicitly available to the system). The second types of feature available in the edge-filtered image are land marks such as large obstacles which can be used to aid robot localization. In this study the colors of the obstacles where known and so were identified using RLE rather than the edge detection techniques. Identifying the horizon means that a long almost horizontal (at least not near vertical) line must be identified. A similar approach will need to be adopted if there are walls or corridors in the image, that is, long lines in the image are identified before further processing.

3 +ve +ve +ve -ve +ve -ve -ve -ve α Fig. 4. x and y axis intercepts for lines of angle and - vertical range horizontal α range a) b) β β Fig. 5. c) Topological view of the polar parameter space, showing continuity of the, boundary and the join of the boundary, where α = -max(height, width) and β = height + width. Fig. 5. a) Maximum x and y axis intercepts, b) Range values in parameter space showing discontinuity at θ =, where α = -max(height, width) and β = height + width. V. HOUGH TRANSFORM OF RLE EDGES Identifying straight lines in an image requires a first order Hough transform to be applied, normally this transfers the image into the parameter space of straight lines, that is from the position (x, y) of the pixel to the parameter space of the lines in the image (m, c), where y=mx+c represents the equations of the lines in the image. Where the lines in the parameter space intersect represents the equations of the lines in the image, this is also the position where there is a peak of data points in the parameter space. This study uses a polar representation of the first order Hough transform (θ, c) rather than the normal gradient intercept (m, c) format. This is so that singularities associated with vertical lines (infinite gradient and intercept with the y axis) are removed. The angle θ in the polar representation is the angle of the line to the horizontal (range from to ) and the intercept c represents the intercept of the line with the x or y axis. The intercept with the y axis is used for - < θ while the intercept with the x axis is used for θ - and θ >. With this parameter space when θ = - the x and y intercepts are equal (see figure 4) so the representation is continuous as the angle changes across this boundary. When θ = the x and y intercepts are additive inverses (see figure 4) which means that there is a discontinuity in the representation as the angle changes across this boundary (see figure 5 b), however with correct implementation of the neighborhood grouping this is not a problem since a topological mapping of the parameter space is possible (see figure 5 c). The polar representation has an additional advantage in bounding the size of the range of values of intercepts by 2 max(height, width) + min(height, width), see figure 5 a & b. Fig. 6. Grouped Edge detected pixels If the polar parameter space has a high resolution it will be necessary to group neighboring peaks in the parameter space so that similar lines in the image are amalgamated into one. A contiguous near neighbor grouping algorithm [14] is applied to the parameter space to combine similar lines in the image, in this way the number of candidate lines in the image are reduced. The peaks in parameter space are used to identify the straight lines in each object in the image. The linear Hough transform is computationally expensive, especially if all the combinations of pixels that have been edge detected are considered. Even if a statistical approach is adopted the computational time complexity can still be high if a large number of pixels are tested to ensure that no edges in the image are missed. This paper proposes modifying the linear Hough transform algorithm, by applying it to the run length encoded image of the edge-filtered image. This requires a class of color identifiers to be supplied for identifying the lines in the edgefiltered image. In this study a class of sharp lines close to white (255, 255, 255) in the edge filtered image and a wide gray band were suitable to detect both the edges of the obstacles and the edges of the horizon. Having run length encoded the edge-filtered image connected lines of contiguous edges are detected as objects. In the example illustrated in figure 3 this is equivalent to three obstacles (note the obstacles in the distance are viewed as a

4 single object using this algorithm since the edge maps overlap) Several vertical lines are detected due to edges caused by rapid variation in the object colors due to shading and shadow effects. Four horizon elements are identified, one of which is very small and is filtered off as noise. Figure 6 illustrates a region of edge detected image that has been run length encoded. The Hough transform of the RLE edges is performed on each object separately. This reduces the computational complexity of the algorithm as interactions between the objects are not added to the parameter space model, reducing the interference effect between the different lines in the image. With run length encoding only the start and end positions of the pixels in each horizontal row are recorded, so random pixels within the run length are selected for the Hough Transform to parameter space. Each obstacle in the example image (figure 3) consists of two long straight lines and two short straight lines. The Hough transform is biased towards the long lines as they provide more candidate points in the parameter space. The two short lines are also identified since they provide two peaks in parameter space after applying the neighbor grouping algorithm. This is the case for the base line of the obstacles even though this line is barely straight. Figure 7 illustrates the selection of the selection of the candidate lines in the given object. The RLE edges of the horizon form 3 objects which are also transformed individually using a linear Hough transform. Each object produces a single candidate line in parameter space after grouping neighboring candidate lines in parameter space. Since we are looking for only one horizon line the three candidate lines are compared to see whether they can also be amalgamated into a single overall candidate line. In this case (figure 3) the three candidate lines provided are collinear and so produce only one candidate line. Given the positions of the obstacles, obtained either from the edge detected image or the original run length encoded image as well as the position of the horizon the robot can be localized. The angle of the horizon gives the rotation of the camera and based on the height of the robot and the flat environment the positions of the obstacles relative to the robot can be calculated. VI. OBJECT IDENTIFICATION The lines and points formed by the intersection of the lines in each object define the boundaries of the objects of interest in the image. The pixels within this boundary are used to calculate the color space range values to be used by the RLE algorithm. See figure 8. The mean value of the color components of the pixels that form the object are calculated so that outlier pixels can be eliminated. Outliers occur near the edges and are not representative of the object under study. Typically, variations of more than 4 from the mean represent outliers. For objects that are larger than 1 pixels plus or minus three times the standard deviation of the pixel values is used to calculate the maximum and minimum range values to be used by the RLE algorithm. For small objects the mean plus or minus 15 is used by the RLE algorithm as the standard deviation may not be reliable. If particular features and landmarks in the environment are known, they can be identified from the objects that have been identified above. VII. RESULTS The standard Hough Transform applied to this problem with out using the RLE and aggregation of contiguous pixels is very slow. This is due to the fact that we need to identify all lines, even short ones in the image. The probability of selecting a pixel for the Hough Transform is given by equation 2. p i = n i / p (2) where p i is the probability of selecting a pixel in the line of interest (i), n i is the number of points in the line of interest and p is the number of edge/line points in the image. The probability of selecting two pixels that are in the same line of interest is given by equation 3. 2 p ii = p i (3) where p ii is the probability of selecting two pixels in the line of interest (i). Equation 4 gives the number of sample pairs that must be taken to reliably achieve the given number of candidate lines from the Hough Transform. Fig. 7. Candidate lines in object Fig. 8. Candidate line intersection and enclosed pixels

5 S ii = n/p ii (4) where S ii is the required number of sample pairs to reliably result in n candidate lines that match the given line of interest (i). The long horizon line will consist of about 32 pixels in a 32x24 image. If the image has about 1 edge/line pixels, then the chance of selecting two pixels from the horizon are (32/1) 2. This means the chance of retrieving the horizon line from the Hough Transform is about 1%. This means we need to take ten pairs of points before we find a candidate line that would represent the horizon. If we want to have at least 5 candidate lines before taking that as a legitimate line in the image we need to take at least 5 sample pairs. For smaller lines, say 5 pixels long, we can see that a significantly larger number of sample pairs are required, 5/ (5/1) 2 = 2,. As the number of edge/line points increase we can see the time the algorithm takes will increase significantly O(p 2 ), where p is the number of edge/line points in the image. This means that for small objects the standard Hough Transform would not be able to update the RLE color space range values regularly. The result of this is that as lighting and objects in the image are changed they are not correctly identified by the real-time RLE system. In the biped robot scenario this results in collisions with moving obstacles and selection of non optimal paths through obstacles. For the RLE augmented Hough Transform the contiguous pixels that form one object are used in identifying the required straight lines. If we take the example illustrated in figure 6, the total number of edge/line pixels is 18, and the shortest line is 3 pixels. This requires only 18 (=5/(3/18) 2 ) sample pairs to reliably identify the 4 straight lines with at least 5 candidate lines. VIII. CONCLUSIONS This paper has presented a real time image-processing algorithm based on run length encoding for a simulated biped robot system. This system can be implemented on real biped robot systems to detect obstacles quickly in the field of view. This paper has also presented and edge detection algorithm that uses a Sobel edge detection algorithm augmented with run length encoding to improve post processing of the image using a linear Hough transform. Although run length encoding and the RLE augmented Hough transform have shown promise, significant research effort needs to be directed at real-time generation the color identifiers used in the RLE component of the vision system. In real world environments with highly varying lighting conditions this dynamic update of color identifiers will be essential. This simulated system used in this study provides clean images and so does not reflect reality where there are often variations in the image due to sensor noise. Localization of the robot relative to the obstacles and mapping the environment with sensor noise requires particle filtering and optimal filtering approaches as discussed in [15 & 16]. Future research will model sensor noise and will require these additional techniques to provide reliable recognition of obstacle positions and localization of the robot. ACKNOWLEDGEMENTS The authors would like to acknowledge the use of Massey University s parallel computer the Helix for the computational experiments that supported the results presented in this paper. REFERENCES [1] G. Sen Gupta, D. Bailey, C. Messom, "A New Colour Space for Efficient and Robust Segmentation", Proceedings of IVCNZ 24, pp , 24. [2] J. Bruce, T. Balch and M. Veloso, "Fast and Inexpensive Colour Image Segmentation for Interactive Robots", IROS 2, San Francisco, 2. [3] C. H. Messom, S. Demidenko, K. Subramaniam and G. Sen Gupta, "Size/Position Identification in Real-Time Image Processing using Run Length Encoding", IEEE Instrumentation and Measurement Technology Conference, pp , ISBN , 22 [4] C. Zhou and Q. Meng, "Dynamic balance of a biped robot using fuzzy reinforcement learning agents", Fuzzy Sets and Systems, Vol.134, No.1, pp , 23 [5] K. Jagannathan, G. Pratt, J. Pratt and A. Persaghian, "Pseudo-trajectory Control Scheme for a 3-D Model of a Biped Robot", Proc of ACRA, pp , 21 [6] K. Jagannathan, G. Pratt, J. Pratt and A. Persaghian, "Pseudo-trajectory Control Scheme for a 3-D Model of a Biped Robot (Part 2. Body Trajectories)", Procs of CIRAS, pp , 21 [7] J. Baltes, S. McGrath and J. Anderson, "Feedback Control of Walking for a Small Humanoid Robot", Proceedings of the FIRA World Congress, 23, Vienna, Austria. [8] C.H. Messom, "Vision Controlled Humanoid Toolkit", Knowledge- Based Intelligent Information and Engineering Systems, Lecture Notes in Artificial Intelligence vol 3213, pp , Springer Verlag, Berlin Heidelberg 24. [9] J.Pratt and G.Pratt, "Exploiting Natural Dynamics in the Control of a 3D Bipedal Walking Simulation", Proceedings of the International Conf on Climbing and Walking Robots, [1] A. Boeing, S. Hanham and T. Braunl, "Evolving Autonomous Biped Control from Simulation to Reality", Proc of 2nd International Conference on Autonomous Robots and Agents, pp , 24. [11] J. Chestnutt, J. Kuffner, K. Nishiwaki and S. Kagami, "Planning Biped Navigation Strategies in Complex Environments", Proceedings of Int ernationalconf on Humanoid Robotics, 23. [12] M. Ogino, Y. Katoh, M. Aono, M. Asada and K. Hosoda, "ision-based Reinforcement Learning for Humanoid Behaviour Generation with Rhythmic Walking Parameters", Proceedings of 23 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp , 23 [13] O. Lorch, A. Albert, J. Denk, M. Gerecke, R. Cupec, J. F. Seara, W. Gerth and G. Schmidt, Experiments in Vision-Guided Biped Walking, Proceedings of IEEE/ RSJ International Conf on Intelligent Robots and Systems, 23. [14] D.C.I Walsh and A.E. Raftery, "Accurate and Efficient Curve Detection in Images: The Importance Sampling Hough Transform.", Pattern Recognition, vol 35, pp , 22. [15] D.C.K. Yuen, and B.A. MacDonald, "Theoretical Considerations of Multiple Particle Filters for Simultaneous Localisation and Map- Building", Knowledge-Based Intelligent Information and Engineering Systems, Lecture Notes in Artificial Intelligence vol 3213, pp 23 29, Springer Verlag, Berlin Heidelberg 24. [16] G. Sen Gupta, C. H. Messom, S. Demidenko, "Real-Time Identification and Predictive Control of Fast Mobile Robots using Global Vision Sensing", IEEE Transactions on Instrumentation and Measurement, vol 54, No 1, pp