An Omnidirectional Vision System for Soccer Robots

Size: px
Start display at page:

Download "An Omnidirectional Vision System for Soccer Robots"

Transcription

1 An Omnidirectional Vision System for Soccer Robots Diplomarbeit von Felix v. Hundelshausen 1 Institut für Informatik, Freie Universität Berlin, Takustr. 9, Berlin April 11, 2001 Betreuer: Prof. Dr. Raúl Rojas 1 hundelsh@inf.fu-berlin.de

2 Abstract This thesis develops a local omnidirectional vision system for the FU-Fighters, the F180 robotic soccer team of the Freie Universität Berlin, playing in the Small Size League of RoboCup. RoboCup was founded in 1997 with the aim to foster the development of artificial intelligence and related field by organizing international games where robotic soccer teams compete. The system consists of a small PAL video camera mounted vertically on top the robots. A concave mirror placed above the camera reflects the 360 degrees of the field around the robot. The image is sent by a radio link to an external computer for processing. The omnidirectional vision system can find the ball and detect the presence of other robots on the field, classifying them as obstacles. The outline of the field is also detected and is used to determine the position of the robot. In order to be able to process 24 fps the movement of all objects is tracked, including the ball, the obstacles, and the outline of the field. For all objects, we divide the image processing task into the two following phases: Initial Search, to locate the objects on the field, and Tracking, to follow moving objects in the video stream. The key of our approach is to search for color transitions along lines. We present an algorithm for this subtask. The omnidirectional vision system is simple and inexpensive. Neither special hardware for image processing nor special optical devices have been used: The concave, parabolic mirror mounted above the camera to obtain the omnidirectional view was taken from a conventional lamp.

3 Contents 1 Introduction Small Size League Global Vision Omnidirectional Vision Philosophy of the Approach Basics Formation of Images Camera Mirror Video Input Perceived Image and Related Difficulties Distance Function Preservation of Angles Reference Points of Objects Excluding Regions from Image Analysis Color Classification Selection Tools Combination of Regions Adding Regions to the Lookup Table Color Transitions Dividing the Task Algorithm CAD-Model Initial Search Radial Method Evidence Accumulation Hough Transform for Wall Detection Detecting Goals Detecting Obstacles Initial Ball Search Region Growing Algorithm

4 3.5 Initial Robot Localization Direct Localization General Localization Tracking Tracking the Playing Field Processing Horizontal Track Grids Processing Vertical Track Grids Combining Displacements and Angles Updating the Position of the CAD-model Registration for Initial Search Tracking Ball and Obstacles Implementation and Results Implementation Results Conclusions 58 2

5 Chapter 1 Introduction After Garry Kasparov was defeated by IBM s Deep Blue Supercomputer in May 1997, forty years of challenge in the artificial intelligence (AI) community came to a successful conclusion. But it also was clear that a new challenge had to be found. By mid-21st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup. 1 This is how the ultimate goal was stated by the RoboCup Initiative, founded in 1997, with the aim to foster the development of artificial intelligence and related field by providing a standard problem: robots that play soccer. It will take decades of efforts, if not centuries, to accomplish this goal. It is not feasible, with the current technologies to reach this goal in any near term. However, this goal can easily create a series of well directed subgoals. The first subgoal to be accomplished in RoboCup is to build real and simulated robot soccer teams which play reasonably well with modified rules. Even to accomplish this goal will undoubtedly generate technologies with impact on broad range of industries. The official first RoboCup games and conference was held in 1997 with great success. Over 40 teams participated (real and simulation combined), and over 5,000 spectators attended. Meanwhile there mainly exist three different leagues: Simulator League: Software agents play soccer using the RoboCup soccer server simulator, available from the RoboCup Web page. Small Size League (F-180): Teams of up to five real robots of small size (approximately 15 cm in diameter) compete on a m 2.74 m field. Middle Size Robot League (F2000): Teams of up to five real robots of medium size (approximately 50cm in diameter) compete on a m 8.22 m field. One problem domain in RoboCup is the field of Computer Vision. Its task is to provide basic information that is needed to calculate the behavior of the robots. There are two 1 3

6 different approaches: Global and local vision. In a global vision system a camera is mounted at some fixed location outside the playing field, observing the game. In contrast, in local vision a camera is installed locally on the robot. Although the computational effort is higher, local vision has the advantage that near objects can be seen more clearly. This is particularly important for controlling the ball. Moreover a robot with local vision will be more flexible as it is not bound to a location where a camera is installed. Especially omnidirectional local vision systems have become interesting in the last years, allowing a robot to see in all directions at the same time without moving itself or its camera. Local Vision is the method used by most teams in the F2000 league. The Golem team [4] impressively demonstrated at the last world championships in Melbourne that, using an omnidirectional camera, sufficient information for controlled play can be collected. Another example for the use of omnidirectional cameras is the goalie of the ART team [1, 2, 3]. In our league, F180, only three teams tried to play with local vision in Melbourne. The main reasons are that due to space and energy constraints smaller cameras of lower quality must be used and that less computing power is available on the robot. Recently, the OMNI team [5] demonstrated controlled play with omnidirectional local vision at the Japan Open competition. This team sends the video stream to an off-the-field computer that employs special purpose hardware for image processing. The main idea in this thesis was to implement a tracking system for the analysis of the video stream produced by an omnidirectional camera that only needs to inspect a small fraction of the incoming data. This allows the system to run at full frame rate and full resolution on a standard PC. The system finds and tracks the playing field, the ball and obstacles. The playing field is tracked because knowing the placement of the playing field from the robot s point of view equals knowing the position and orientation of the robot. The field is tracked by consecutively registering a CAD-model to the video stream. This is done by searching color transitions along lines perpendicular to edges that are specified in the CAD-model and by consecutively adjusting the model to the found transitions. The adjustment consists of a translation and a rotation that are calculated by comparing the edges in the CAD-model and the corresponding color transitions. To initially localize the robot two different methods are presented. The first is a direct calculation that is specialized to be used in the RoboCup environment. The second is more general and flexible since it uses the CAD-model as only specification of the environment. It uses an accumulation technique to determine some kind of plausibility for several hypothetical positions and uses the tracking mechanism to verify the positions. For finding the goals, the walls and the obstacles lines are stretched out radially from the center of the image and color transition from the green floor to the respective colors are searched for. To detect the ball a region growing algorithm will be presented that finds clusters of a specific color. For tracking the ball and the obstacles the same region growing algorithm is applied but it is restricted to a small rectangular area around the predicted position of the respective object. For color classification a lookup table is used. The remainder of this introduction describes the Small Size League of RoboCup and the difference between global and local omnidirectional vision. It concludes with scratching the philosophy of the presented approach. Next, a chapter follows that describes some 4

7 basics that will be used later. It encompasses the optical mapping, the color classification, the search for transitions along lines, and the CAD-model. Two chapters follow that detail initial search and tracking respectively. Finally, the results and future goals are described. 1.1 Small Size League In Small Size League two teams each consisting of five robots compete. They have a size of approximately 18 cm in diameter and can be constructed freely, solely underlying some restrictions made by the RoboCup Initiative. The playing field has a size of cm 274 cm. The floor material consists of green felt mat or carpet. The walls around the field are painted white, the goals blue and yellow. The ball is an orange golf ball. Figure 1.1 shows two teams of robots competing against each other. Figure 1.1: Small Size League. Two teams of robots are competing in Small Size League. 1.2 Global Vision In Small Size League most teams use a global vision setup, which is illustrated in figure 1.2. A camera is mounted above the playing field. It is connected with a computer running special software developed individually by the different teams. The software analyzes the images of the camera and finds the positions of the ball and the robots which have color markers to simplify the recognition and identification. After the behavior of the robots has been calculated, the computer sends back control commands to the robots, using radio transmission. So the robots are remote controlled by computers and not by humans, as many people believe, seeing it for the first time. Figure 1.3 shows a Classic -robot of the FU-Fighters that is employed when global vision is used. 5

8 Figure 1.2: Global vision setup. playing field. Each team has its own camera mounted above the 1.3 Omnidirectional Vision For RoboCup 2001 we decided to develop a new generation of soccer robots based on local vision where each robot carries its own camera. This is the method used in the mid-size league, but implementing local vision in our league is still a challenging problem for the following reasons: a) the cameras that can be used are of lower quality, therefore the software used for the analysis of the images must be robust; b) the whole system must be kept small; c) there are also hard constraints on the amount of power and hardware that can be used. For our omnivision system we decided to use the chassis of the previous generation of robots and to retrofit it with a small video camera and a mirror. We tested several different configurations and different types of mirrors and we found out that an inexpensive concave mirror could be used with good results. Figure 1.4 shows one of our Classic robots with an omnidirectional vision device incorporated on top. We mounted a small PAL camera on top of the robot directed upwards and looking directly into a parabolic mirror 2. The parabolic mirror collects light rays from all directions, reflecting them directly into the pinhole of the camera. Before settling for this mirror, we tried to use a convex spherical mirror and a mirror cut as a cone. The concave mirror yields better results because the parabolic shape closely approximates an ellipse. The ellipse has the property that light rays going through one focus and reflecting on the mirror, go through the other focus. If one places the pinhole of the camera at the focus farther away from the reflective portion, then we obtain a good focused image regardless of the distance of the objects. The edge of the reflective side must be cut at the level of the other focus, so that an image of the plane from the current position to infinity can be obtained. Figure 2 We cut the reflector of a flashlight to the appropriate length. 6

9 Figure 1.3: The Classic -Robot of the FU-Fighters. There are markers on the top of the robot, enabling the global vision software to identify the robots. 1.5 shows how light rays are reflected into the camera. In order to avoid carrying a large computer, the images of the video camera are transmitted to a computer through a radio link. Each robot with omnidirectional vision has therefore two radio connections: One for the transmission of control commands and one for the transmission of video images. While the computation of the behavior of all robots is fast enough to be performed with one single computer, processing the local vision data requires one extra computer for each robot. This means that the vision computers must be connected to the computer calculating the behavior through a local area network. Figure 1.6 illustrates this setup for two robots. Once the image from the local camera has been transmitted it must be processed at the off-the-field computer. The first problem is for the robot to compute its own position on the field. Even if the robot sees the ball and can move towards it, the actual path depends on the current position: if the ball lies between the robot and the own goal, then the robot must come from behind. If the ball lies between the robot and the enemies goal, then the robot can just follow a linear path towards the ball. This means, that for planing the path to the ball, it is also important to know the positions of the goals, at least relative to the ball. Whenever possible the exact position of the robot on the playing field should be determined. Furthermore, for planning a path, it is important to know where obstacles are located. These obstacles might be opponents or teammates. Therefore the following three tasks have to be accomplished: Detecting the ball Localizing the robot Detecting obstacles 7

10 The main difficulty is doing all this in real time, that is, the data must be processed at a rate of at least 20 frames per second. With the resolution of our camera of pixels and two bytes per pixel, there is a data throughput of about MB per second. However, it is not necessary to look at all this data in every frame, because individual frames are not independent of each other. If we know the position of an object in one image it is very likely that the same object will be at a near location in the next image. To exploit this fact we have divided the visual perception of the system into two phases. One phase is the initial search, which tries to find the positions of the objects for the first time or when recovering from failure. The second phase, tracking, starts after the initial search was successful and tries to follow moving objects in the images. If at some point tracking fails, initial search will be restarted. 1.4 Philosophy of the Approach Thinking of vision as a constructional process as described in [8], the computer vision community has spent 30 years of enormous research effort to define general low-level processes that find significant features in images and construct objects from them. But up to now this typical bottom up approach did yield little success. The difficulty lies in recovering features undamaged and free of breaks, and in successfully grouping them according to the object to which they belong. A new approach was taken in thinking of vision as a top-down process, where prior information about the visual scene is imposed from start. The first contribution towards this direction was the snake, for which the original paper is [10]. This approach has been extended in many ways, where they all use some kind of model that adjusts itself according to the image. In this work also a specific model will be used to track the outline of the playing field, in order to know the robot s position at any time. 8

11 Figure 1.4: The Classic -Robot using omnidirectional vision. A small camera is directed upwards looking into a parabolic mirror that reflects the 360 degree of field around the robot. Figure 1.5: Optical Path. Principle of how rays of light are reflected into the camera. Figure 1.6: Omnidirectional Vision Setup. Setup, using two robots with omnidirectional vision. 9

12 Chapter 2 Basics This chapter describes some basics that will be used later for initial search and tracking. It starts with the optical mapping of the omnidirectional vision, which concernes the relationship between the positions of objects in the real world and their positions in the robot s perceived images. 2.1 Formation of Images To understand the relationship between world and image coordinates that arises from the combination of the camera and the mirror we first have to consider the camera by itself Camera Figure 2.1 shows the camera 1 we use. Figure 2.1: Camera.The PAL-camera has a resolution of pixels. How are the images of the camera formed? To gain an insight we can imagine light to consist of single rays. Assume there is only one source of light, for example a lamp. 1 It can be obtained from Conrad Electronics (order number: ) 10

13 Figure 2.2: Correspondence between pixels and rays. Each pixel corresponds to a ray that can be determined by finding a location on the wall that corresponds to the pixel. This lamp can be thought of emitting light rays in many different directions. These rays interact with the objects they hit: They might be reflected, refracted or scattered, before only few of them enter the camera and hit a receptor. The activations of the receptors form the image, consisting of pixels. Now we can invert this process and ask: Which rays that enter the camera are mapped to a specific pixel? A simple experiment informs. We pose the camera in front of a wall, as depicted in figure 2.2, and mark the location on the wall that corresponds to the specified pixel. The location can simply be found by moving a scrap of black paper along the wall, until the specified pixel becomes black. So each pixel in the image can be identified with a ray that is determined by P, the focus of the camera and the angles α and ϕ, according to figure 2.2. P is given and α equals the angle α in the image as illustrated in the left part of figure 2.2. Furthermore we measured a simple linear relationship between ϕ and d for the camera we used, ϕ = εd (2.1) where ε is a constant. To observe this relationship, we can specify n pixels with different distances d 1, d 2,..., d n to the center of the image and determine the angles ϕ 1, ϕ 2,..., ϕ n 11

14 Figure 2.3: Linear relation. There is a linear relation between the distance of a pixel from the center of the image and the angle of the corresponding ray to the normal of the camera. respectively. Figure 2.3 shows their linear relation. Knowing P, α, and ϕ, we are able to predict the ray that corresponds to a specific pixel Mirror The concave parabolic mirror we use was taken from a conventional lamp. We want to determine the height at which the mirror should be located above the camera in order to maximize the field of view. This means that the outmost rays of the camera should be reflected horizontally by the mirror in the borderline case. Unfortunately, if the mirror does not have the appropriate proportions, it can happen that (reversing the path of light) the outer rays of the camera are reflected into the mirror itself (Figure 2.4). To avoid this, the mirror has to be cut. But what is the appropriate depth of the mirror? Consider figure 2.5 for the following calculation. The shape of the mirror is given by the parabola f: f(x) = ax 2 + c (2.2) The constant a is given, but c is unknown. The outmost ray r of the camera is specified by r(x) = mx (2.3) because the coordinate system is located in the focus of the camera. The slope m of the ray can be determined by calculating the angle ϕ (according to figure 2.2) that corresponds to one of the outmost pixels that should see the mirror. If the resolution of the image is x y and x > y then the distance of such a pixel to the center of the image is y, since the image should contain the mirror as large as possible. The angle ϕ can be 2 calculated by equation 2.1. Then the slope of the ray is: m = tan( π 2 ϕ) = tan(π 2 ε y 2 ) (2.4) 12

15 Figure 2.4: Self-Reflection. A ray that is reflected into the mirror itself (reversing the path of light). Figure 2.5: Combining Camera and Mirror. Calculation of the appropriate depth and position of the mirror. 13

16 Since we know that the reflected ray 2 is orientated horizontally in the borderline case, we can calculate the slope m t of the tangent t at the point where the ray r and the parabola f intersect. To perform this calculation we define the vector m corresponding to the slope m of ray r: ( ) 1 m = (2.5) m Next, we specify the horizontal vector m r of the reflected ray to have the same Euclidean length as m: ( ) 1 + m 2 m r = (2.6) 0 Considering the parallelogram in the right part of figure 2.5, yields the vector s which specifies the direction of the axis of reflection. s = m m r = ( ) m 2 m (2.7) Since the tangent t is orthogonal to s, the direction m t of the tangent is: ( ) m m t = m 2 (2.8) Transforming this vector to the corresponding slope m t of the tangent yields: m t = y x = m 2 Next, we calculate the location x s where the parabola has the same slope: df dx (x s) = m t m 2ax s = m 2 m x s = m 2 2am This directly yields the desired depth of the mirror. (2.9) (2.10) d = ax 2 s (2.11) The height c at which the mirror has to be mounted above the focus of the camera then is given by: c = mx s + d (2.12) 2 For optical analysis it is common to reverse the path of light. 14

17 In our case, where m 1.87 and a 0.523, the results are: x s cm d 1.33 cm c 4.31 cm (2.13) 15

18 2.1.3 Video Input The capture driver has been configured to write the images in 15-Bit RGB format into host memory. Here one pixel is stored in two bytes, where five bits are used for the red, green and blue component respectively. The most significant bit is not used. Figure 2.6 shows the bit assignment of one pixel. This format has two main advantages: Firstly Figure 2.6: Bit assignment of one pixel. The pixels are stored in RGB format. Five bits are reserved for the red, green and blue component respectively. Bit 15 is unused. the complete color information can be accessed by one processor instruction which can be performed very fast. Secondly we obtain only 2 15 = possible colors which will be important for fast color classification. The size of one image is 768 x 568 pixels, where the pixels are stored consecutively line by line within host memory Perceived Image and Related Difficulties Figure 2.7 shows an image that was perceived when the robot was standing at a corner of the playing field. According to the calculation above, the mirror takes up the whole image. The point that is located in the mid of the mirror is called perception origin in the following. It will be used to measure distances and angles. In figure 2.7 there are markings for calibration. Normally, these markings are not present. They are used to establish a relationship between distances in the image and distances in the real world, as described in the next section. There are several problems, concerning the image: The sticks that hold the mirror occlude parts of the visual field. Furthermore they can easily be confused with obstacles in the image, as their color is similar. It is not possible to focus near and far objects at the same time. In figure 2.7 the area near the blue goal is out of focus. By cutting the mirror the quality of the outer regions is impaired. As a result, far objects sometimes are distorted. In figure 2.7 this can be observed for the wall near the blue goal. Some pixels have false colors. For example, some pixels between the green floor and the white wall are blue, although there is no blue object in the world. These false 16

19 Figure 2.7: Perceived Image. colors are caused by the camera, as the color receptors for a specific pixel are located at different positions. This will complicate the search for color transitions later. Due to the radio transmissions the quality of the image suffers. interferences as depicted in figure 2.8 occur. Sometimes even Distance Function How is an object mapped into the image? Consider the situation in figure 2.9, where an object is located in a distance d to the robot. In the robot s image the edges b and c are mapped to a greater distance to the perception origin than the edge a. We are interested in the function f, that maps the distance d in the real world to the pixel distance f(d) to the perception origin in the image. Here we agree, that with distance to an object, we actually mean the distance to the bottom most part of the object. This function f will be called distance function in the following. Although it is possible to derive an analytic expression for this function, the resulting term would be complex. For practical use it is sufficient to measure some samples and to establish a function through interpolation. Therefore objects (i.e small scraps of paper) are placed in specific distances to the robot. These markings for calibration can be seen in figure 2.7. By measuring the corresponding pixel distances in the image and using some kind of interpolation, the real camera function can be approximated. Since the mapping is bijective we can easily calculate the corresponding inverse distance function. To give an example for the use, assume that we have recognized the yellow goal in the image and have determined its pixel distance to the perception origin. Then we can calculate its distance 17

20 Figure 2.8: Interference. Interference caused by radio transmission. in the real world by the inverse distance function. Figure 2.10 shows the two functions, sampled with 6 measurements Preservation of Angles Angles are preserved by the optical mapping. This means that the angle between two lines, reaching from the robot to different objects, is the same as the angle in the image that is formed by the lines from the perception origin to the mapped objects. This fact will be used for localization later. Figure 2.9: Mapping objects. Higher edges are mapped to farther pixels. 18

21 Figure 2.10: The two distance functions Reference Points of Objects To determine the distance to an object as precisely as possible, it is important to recognize a point that is near the floor, because the camera mapping is only correct for this height 3. On the other hand it is important to find the horizontal mid of the objects to specify their position correctly. Figure 2.11 illustrates which points of objects should be found. These points will be called reference points in the following Excluding Regions from Image Analysis There are certain regions within the perceived images that have to be excluded from analysis. One of them is the part in the image that reflects the robot itself. Other regions are the sticks that hold the mirror and the areas outside the mirror. For that a boolean array of the same size as the image has been used to specify for each pixel whether it should be regarded as part of the visual field or not. Figure 2.12 shows the regions that have been excluded. All algorithms that access the image check this array to see whether a specific pixel belongs to the visual field. For instance, when tracking obstacles black clusters are searched for within certain subregions of the image. But black pixels of the sticks will not be found because they are excluded from the visual field. To specify the excluded regions, special software has been developed. 3 Although it is possible to extend the camera mapping to be applicable for all heights the problem only is converted into the problem of determining the height of the found part. 19

22 Figure 2.11: Reference Points. The white crosses mark the points that should be found to determine the distance to the objects and to specify their positions. 2.2 Color Classification Image analysis in the RoboCup domain is simplified, since objects are color coded. Black robots play with an orange ball on a green field that has yellow and blue goals and white walls. Thus, a pixel s color is a strong hint for object segmentation. We exploit this fact by defining color classes, using a look-up-table (LUT) for fast color classification. The table consists of entries, each 32 bits wide, occupying 128 KB in total. Each bit expresses whether the color is within the corresponding class or not. This means that a certain color can be assigned to several classes at the same time. To classify a pixel, we first read the pixel s color and then use the color as an index into the table. The value (32 bits) read from the table will be called color mask of the pixel in the following. In the next sections especially the sequence of color masks of pixels along a line are investigated. Figure 2.13 shows such a line. To illustrate the classification of the pixels along a line a certain diagram is used that is shown in figure There are six color classes that are named by the objects they represent: Floor, Wall, Ball, Obstacle, Yellow and Blue Goal. The naming may be misleading in some cases, since it can happen that a pixel is dark and is therefore classified as Obstacle, although the pixel does not belong to an obstacle but to a shadow for example. For all pixels along the line their colors are painted at the bottom of the diagram, and above the first six bits of their color masks are indicated by black pieces of lines. For each color mask, bit 0 corresponds to the class Floor, bit 1 to class Wall,... If a bit is set, then a black piece of line is drawn at the appropriate position. To define color classes specific selection tools have been implemented. The user 20

23 Figure 2.12: Visual Field. The black regions are excluded from perception. Figure 2.13: Line along which the pixels will be classified in figure defines regions within the image by making use of them and then adds the colors of the pixels of the region to a specified color class. There are different selection tools and different mechanisms how consecutively selected regions are combined to form a new composed region. Besides defining color classes, the selection tools are used to specify the regions that are excluded from visual processing Selection Tools Each selection tool selects a subset of pixels of the image. The following tools have been implemented: Line Selection: Pixels along a line are selected. Rectangle Selection: Pixels on a rectangle are selected. 21

24 Figure 2.14: Classification of the line in figure Ellipse Selection: Pixels on an ellipse are selected. Area Selection: Pixels within an area of similar colors are selected. The Area-Selection tool is the most useful. The algorithm that chooses the pixels is a region growing algorithm that starts with a seed pixel and iteratively adds all surrounding pixels whose color is not too distant from the color of the seed pixel. In this work the Eucleadean RGB distance has been used, but it can easily be exchanged by other distance measures if necessary. The algorithm will be described in detail later, when it is used in a slightly modified version for initial ball search. Figure 2.15 shows a region within the yellow goal that has been selected by this tool. Figure 2.15: Area-Selection. All surrounding pixels that have a color distance to the seed pixel that is below a certain threshold are selected. The user specifies the seed pixel by using the mouse. 22

25 2.2.2 Combination of Regions There are four different modi how consecutively selected regions can be combined. In the Equal-mode the recent selection replaces the actual region. In the Plus-mode it is added to the actual region, in the Minus-mode it is subtracted and in the Section- Mode the section of both sets of pixels is calculated. Figure 2.16 illustrates how two different regions are added. Figure 2.16: Adding two regions. By the combination of regions complex selections can be performed Adding Regions to the Lookup Table When a region has been selected the colors of all pixels within this region can be added to the specified color class. Rather than only adding the color of each pixel to the LUT, all colors that lay within a small sphere around each pixel are added, thinking of a color as a point in RGB space. 2.3 Color Transitions Initial search and tracking, both will base on the capability to find specific color transitions along a line of pixels within the robot s images. For instance, when looking for the yellow goal, we will seek transitions from the green color class of the floor to the yellow color class of the goal. This seemingly simple task becomes complicated by the following facts: It can happen that between the transition of two color classes there is a third color class. This is often caused by errors due to the camera. Figure 2.17 depicts such a case where between the green of the floor and the white of the wall there is a small blue streak. We only want to find transitions, whose corresponding color areas have a minimum specified width. For instance, when searching for the walls, we don t want to find the white marking lines, when looking for a transition from green to white. By demanding a width that is greater than that of the marking lines, only the transitions to the walls will be found. It must be fast, since it is often used in later algorithms. 23

26 Figure 2.17: False Colors. Blue interference between the green floor and the white wall Dividing the Task The task can be divided into two parts. The first part, Class Encoding, has the task to iterate through the pixels along a line and to classify them. The second part, Transition Search, has the task to find the specified transition. The algorithm can be generalized to find transitions on circles, curves or any sequence of pixels, by exchanging the first part Algorithm Figure 2.18 depicts an ideal transition from color class a to color class b. We note that the Figure 2.18: Ideal color transition. pixels of class a in figure 2.18 might have different color masks, but the bit corresponding to class a is set for all of them. The transition is ideal because the area A and the area B are not interrupted by foreign class masks, and directly adjoin. In practice, this ideal case rarely occurs. Situations as depicted in figure 2.19 are more likely. Assume we have a hypothetical location t {0,..., n} Figure 2.19: Color transition in practice. of transition. The expression t = 1 would specify a transition between the first and the 24

27 second pixel. We define two areas, directed to both sides of the transition t, I a (t) := {t n a,..., t 2, t 1} {0, 1,..., n 1} I b (t) := {t, t + 1,..., t + n b 1} {0, 1,..., n 1} where n a, n b > 0 are the predefined widths of the areas, as depicted in figure Figure 2.20: Inspection areas. transition. The inspection areas help to define the presence of a Let A be the set of all color classes and let m i A be the color mask of pixel i, where i = 0, 1,..., n 1. Let a, b A be the color classes of the searched transition. The numbers of pixels c a (t), c b (t) within each inspection area having the appropriate color mask are given by: c a (t) = {i I a (t) a m i } c b (t) = {i I b (t) b m i } Next, we define, that a transition is present at position t, if and only if c a (t) and c b (t) are greater or equal to some predefined minimum numbers min a, min b IN 0. The task now is to calculate t. A first approach is to increment t stepwise, starting from zero, and to calculate c a (t) and c b (t). Rather than counting the number of appropriate color masks within each inspection area by looking at all elements again and again it is sufficient to look at two pixels respectively for updating c a,b. Figure 2.21 illustrates this. The algorithm can be stated by the following pseudo code: Determine c_a and c_b for t=0; FOR t=0...n DO BEGIN Update c_a and c_b by investigating the "border pixels" of the inspection areas; IF c_a>=min_a AND c_b>=min_b BEGIN A transition has been found; BREAK; END END 25

28 Figure 2.21: Updating the numbers c a,b. Here the inspection area I b is observed while moving one step to the right. To update c b (t) it is only necessary to look at the pixel that comes in and the one that falls out. The incoming pixel has color class b, and so c b is incremented, but since the pixel that falls out also has color class b, c b remains unchanged. The running time of the algorithm clearly is O(n), where n is the number of pixels. Combined with an assembly coded Bresenham algorithm [6] for the classification of pixels along a line over lines at a length of about 300 pixels can be analyzed per second on a 900 MHz Pentium III processor. 2.4 CAD-Model Rather than developing a vision system that is only applicable for RoboCup the idea in this thesis was to develop a system that works with a whole class of environments that can be defined by a special CAD 4 -model which specifies the shape and the expected colors of the environment. The software system should be able to localize and track the robot s position without any prior knowledge than the model. Therefore a CAD software part has been developed, enabling the user to specify an environment. The construction of elements can comfortably be performed with the mouse. Scrolling and zooming is possible and OLE2 drag and drop has been implemented. The architecture of the CAD-system is object oriented. An important idea was that the elements in the CAD-model not only model the environment but also specify the way how they can be initially localized and tracked in the video images. For instance, during general localization each CAD-element has a method that is called when a certain image feature has been found. The method then marks areas on a specific accumulation grid where the robot can be inferred to possibly be. If sometime new elements have to be constructed to model a different environment a new element class can be derived that overwrites the method that infers possible positions of the robot from the found feature. 4 CAD=Computer Aided Design 26

29 For modelling the playing field of RoboCup a specific CAD-element plays a central role: Lines that separate different color areas. For instance, the long edges of the playing field are modelled by lines that separate the green floor from the white walls. Hence, for each line in the CAD-model two color masks are stored, which specify the expected colors for each side of the line. Figure 2.22 illustrates the model that has been defined for the playing field of the Small Size League. The lines with the colors green and white represent the Figure 2.22: CAD-model. Each line is drawn by two color streaks. In reality the streaks are not part of the model but just each line between them together with two variables storing the color masks. outline of the playing field. The vertical lines model the goalposts. During initial search and tracking the CAD-model will be registered to the video images by seeking for color transitions orthogonal to the lines of the model, where the colors of each line specify the transition colors searched for. 27

30 Chapter 3 Initial Search 3.1 Radial Method The Radial Method stretches lines radially from the robots s perception origin and searches for transitions from the floor to the specified color class of an object. Figure 3.1 shows the result of searching the transitions from the floor to the walls. This method Figure 3.1: Radial Method. Seeking for the transitions to the wall. not only can be used to find the walls, but also to find the goals and the obstacles. The advantage of this method is that it finds parts of the objects that are near the floor, which is important to calculate the correct distance to the objects. 28

31 3.2 Evidence Accumulation Within this work a certain technique occurs at several places. It is used to find the best among many interpretation of an observation by accumulating indications for each possibility. It is used for the detection of lines, in form of the Hough Transform. for initial goal and obstacle detection for initial robot localization for combining angles during tracking The technique can be seen as some kind of fuzzy histogram. Measured or generated data is grouped into bins, each having the same width. The bins result from discetizing a one or two dimensional continuous space. The data is accumulated by counting how many elements fall into the bins respectively, but in contrast to a histogram, where each data element only corresponds to a single bin, a fuzzy membership is applied to increase the counters. For a single data point x the accumulation counters of all bins are increased by the belief that x belongs to the respective bin. This belief is expressed in terms of the distance d to the bin where x directly falls into. There are many possibilities for a membership function u(d) which gets the values between [0, 1]. In this thesis Gaussian shapes have been used to model them. Figure 3.2: Comparing a usual and a fuzzy histogram. histogram is spread over several bins using a Gaussian shape. Each entry in the fuzzy Figure 3.2 illustrates the difference between a usual histogram and a fuzzy histogram, assuming a discretization of [0, 4] into 40 cells and the two data points x 1 = 0.55, x 2 =

32 The advantage of using a fuzzy histogram emerges when mixture of similar values is desired. For instance, if x 1 = 0.55 and x 2 = 0.75 figure 3.3 shows how the accumulation in the fuzzy histogram forms a maximum in the mid of x 1 and x 2, whereas in the usual histogram no mixture occurs. By adjusting the variance of the membership function one can specify the distance up to which data mixes. Figure 3.3: Mixture of data. The advantage of a fuzzy histogram is that similar data mix Hough Transform for Wall Detection The Hough Transform [9] is a standard tool in image analysis that was primarily used to detect straight lines but has been extended to detect any geometric shape that can be specified by a fixed number of parameters. Usually it is only applied for shapes (i.e. lines) that are determined by two parameters. It uses a discretization of the according parameter space into bins to accumulate possibilities of parameter combinations. Hence, the parameter space can be seen as a two-dimensional histogram. The task of recognizing a shape in an image is transformed to the task of recognizing a point with maximal accumulation in parameter space. Its eminent use emerges when the shape one is looking for is sparsely digitized, has holes or the image is noisy. According to figure 3.4 a line can be parameterized in the form ρ = x cos φ + y sin φ (3.1) When a point is found in the image that is supposed to be part of a line, then a sinusoidal curve in parameter space corresponds to all lines that pass through this point in image space. By accumulating the curves of several points that lie on a line in image space, the parameters of the line can be obtained by searching for the bin in parameter space that 30

33 has the maximal accumulation. Care has to be taken when one quantizes the parameter space. If the quantization is too coarse, nearly parallel lines which are close together will lie in the same bin. On the other hand, if the bins are chosen too fine, each intersection of two sinusoidal curves can be in a different bin. To avoid this, one can extend the entries to surrounding bins as described above, using a Gaussian-like distribution for example, instead of accumulating to single bins. Figure 3.4: Hough Transform. The curves in parameter space correspond to the points in image space. They intersect at a point which parameters specify the line in image space. To give an example that will be used for initial robot localization later, suppose that we want to find the best visible wall of the playing field. Therefore we first determine several points of the walls by applying the radial method, searching for transitions from the green floor to the white wall. Figure 3.1 shows the found points. All the found Figure 3.5: Parameter Space. The maxima correspond to the three walls in figure 3.1 points are transformed to world coordinates, using the inverse distance function, and the corresponding sinusoidal curves are accumulated in parameter space. Figure 3.5 shows the result of accumulation, where the parameter space has been discretized into bins. The three local maxima correspond to the three walls in figure 3.1. Later we only need to determine the best visible wall. Hence, we will choose the point in parameter space, that has the maximal accumulation. 31

34 3.2.2 Detecting Goals To find the goals we first apply the radial method. Figure 3.6 shows the result of searching the yellow goal. We observe, that two clusters of points have been found, caused by the Figure 3.6: Goal Detection. Seeking for the transitions to the yellow goal. The white bullets mark the found transitions. goalkeeper which occludes parts of the goal. We are interested in determining the reference point of the goal. If p 1, p 2,..., p n denote the found transition points, then ϕ 1, ϕ 2,..., ϕ n denote the angles of the corresponding transition lines and d 1, d 2,..., d n the distances to the perception origin. To mix the angles and the distances we use the accumulation technique described above respectively. This yields a pair (ϕ, d), the estimated reference point of the yellow goal. In figure 3.7 this position is marked by a white cross. The center of the goal has been found, although it is occluded by an obstacle. To demonstrate the power of the accumulation technique, a yellow color blob has been inserted to the image at a location that is different from the goal. But giving this false hint does not change the result. Again the correct position has been found (Figure 3.8). This is, because the transitions of the false hint do not accumulate with the transitions of the goal. Figure 3.9 shows the accumulations of the distances and the angles for the given case. The accumulation technique ensures that near angles are mixed on the one hand, but that spurious transitions do not disturb the mixture on the other hand Detecting Obstacles The detection of obstacles again starts by applying the radial method. But, to obtain multiple positions one can not accumulate the angles and distances using a one-dimensional 32

35 Figure 3.7: Result of goal detection. The center of the goal has been found due to the mixture of the angles using the accumulation technique. discretization respectively, since the correspondence between angles and distances is not known. Instead a two-dimensional discretization is used where the dimensions are the x and y-axis of the robot s local view. Figure 3.10 illustrates this. Searching the maxima yields the result in figure The method that is applied to extract the maxima is described later, when it is used again during intitial robot localization. 33

36 Figure 3.8: False Hint. Although a false hint was given, the correct position of the goal has been detected. Figure 3.9: Result of accumulation. The angles and distances of the transition points are accumulated respectively. The values of the goal have the maximal accumulation. Hence, the spurious transitions do not disturb the result. 34

37 Figure 3.10: Obstacle detection. Left: The transitions from the floor to the obstacles have been found. Right: The positions of the transition points are accumulated in a discretization of the robot s local coordinate system. Single entries have the shape of Gaussians. Figure 3.11: Result of obstacle detection. The white crosses mark the found reference points. 35

38 3.3 Initial Ball Search Finding the ball is of essential importance in RoboCup. Although it seems to be an easy task, detecting the ball clearly demonstrates the difficulties of computer vision. Figure 3.12 shows some images of the ball. Figure 3.12: Appearances of the ball. The ball s appearance varies in color and shape. The simplest idea is to look at all pixels of the image and to stop when a pixel has been found having the color class of the ball, anticipating that the pixel belongs to the ball. But of course this method is not applicable since there are many pixels that are not part of the ball but have its color class. In figure 3.13 all these pixels have been marked with white bullets. The next step is not to rely on single pixels but to look for clusters of pixels. Figure 3.13: Ambiguity of color information. Pixels that have the color class of the ball are marked with white bullets. To find these clusters the accumulation method could be used but a faster region growing technique can be applied, that determines the number of connected pixels for each cluster. The algorithm, detailed in the next section, starts with a region consisting of a single seed pixel at the beginning, having the color class of the ball, and investigates all its neighbors. 36

39 If they have the appropriate color and have not yet been assigned to any region, they are added to the region and again their neighbors are investigated respectively. Applying this algorithm to all initially found ball pixels and only starting the algorithm when the actual pixel has not yet been assigned to any region, yields the sizes of all regions. Here it is assumed that the ball only consists of one region, which is not always true. Fortunately, in the cases where this assumption fails, the ball has a large extent in the image and even the subregions are greater than any other region. Furthermore the region growing algorithm can easily determine the border pixels of the region and therefore chooses the one that is closest to the perception origin. Thus the distances can be calculated precisely. But even knowing the size of the clusters is not sufficient, because the ball can be very small in the image and other spurious clusters can be greater. These spurious clusters are mainly caused by color markers of other robots, as illustrated in figure Figure 3.14: Confusing the ball with color markers. Color markers of robots might have regions with the same color class as the ball. Hence additional information is needed. If not only the position of the robots but also their exact shape in the image was known, then clusters that lay within the shape of the obstacles could be discarded, but unfortunately the exact shape is not known. This addresses the problem that the interpretation of features detected in the image depends on the context. The next idea is to predict the size of the ball in the image and to inspect pixels on a circle around the found cluster, with a radius being a little greater then the predicted size. It was expected that for spurious clusters only few pixels on the circle would have the color mask of the green floor. For the real ball the contrary was expected. Unfortunately this often is not the case, since color markers often are near to green pixels in the image and the ball often has shadows that reduce the number of surrounding green pixels. But one observation helps to identify spurious clusters: Consider the line that reaches from a pixel of the spurious cluster to the perception origin. Since the color markers are on the top of the robots and the sides of the robots are black the line passes first through the black sides before it reaches the green floor. For the ball there is a similar effect, caused by shadows, but here the dark parts on the line are much smaller. Hence, the idea is to look for a transition from the color class of the ball to the color class of the floor, allowing small parts of other color classes (shadows) between them. Thus the detection of the ball consists of three steps: 37

40 Determine all clusters having the color class of the ball and their sizes. Discard all clusters for which no transition to the floor can be found. Choose the biggest cluster. 3.4 Region Growing Algorithm Let P be a m n-matrix with the entries p yx being the pixels of the image. Furthermore let m(p yx ) be the color mask of pixel p yx that can be retrieved by the look-up-table described above in the Color Classification section. Given a specific color mask m s which can be seen as a set of color classes, we can construct a graph G ms = (V, E ms ) from the image, where V := {p yx y = 0, 1,...m 1; x = 0, 1,...n 1} consists of all pixels. To define E ms consider a single pixel p yx. This single pixel has four possible edges e 0,1,2,3 (p yx ) to neighbored pixels (left, right, up, down). If p yx is a border pixel, there are less than four possible edges. Each edge is included in the set E ms, if the neighbored pixel includes the color mask m s. Here includes means, that the pixel must have a color mask that encompasses all color classes of m s. Adding the edges of all pixels in the described manner yields E ms. Figure 3.15 illustrates how G is constructed. Figure 3.15: Constructing the graph G. On the left all pixels that include the specified color mask m s are painted gray. Starting with a seed pixel a P and a color mask m s the goal now is to determine the border pixels of the surrounding region, given by all nodes of G ms that are connected with a. Figure 3.16 illustrates this. The algorithm just works as an iterative breath-first search starting at a. An array V of size m n to mark pixels that have been inspected and a queue q are maintained by the algorithm. Initially all elements of V are zeroed and the queue only stores a. The algorithm repeatingly extracts a pixel from the queue marking 38

41 Figure 3.16: Border pixels. The goal is to determine the border pixels, marked by the black bullets. it as visited in V and investigates its neighbors. If they have not been visited yet and are adjacent according to the definition of G, they are added to the queue. If a pixel has a neighbor that is not adjacent in the sense of G, then it is a border pixel. For those pixels the Euclidean distance to the perception origin is calculated and the nearest of them is maintained. The size of the region can easily be obtained by counting the number of pixels that are extracted from the queue. The running time of this algorithm can be reduced to O(k) where k is the size of the region. For that the array V is only zeroed once and a certain counter c is increased each time the algorithm is run. For marking an element as visited the corresponding element of V is set to the value of this counter. Hence an element is not marked, if its value is below c. By incrementing c before running the algorithm all pixels are implicitly unmarked. Using four bytes for each element of V the algorithm can be run 2 32 times before V has to be zeroed again, which will not happen in practice. 3.5 Initial Robot Localization The task of Initial Robot Localization is to determine the robot s position and orientation on the playing field, given an image of the robot s perception. Although there might be approaches, where the robot does not know its position on the playing field, but just detects objects as the ball and the goals relative to itself, we think that a precise localization is important for intelligent playing. Assume the following situation: The robot does not see the ball and wants to move to the center of the playing field, in order to have a good overview. But if the robot does not know, where it is, it cannot move to the center. Two different methods have been developed for localizing the robot. The first is fast but is only applicable when both goals can be found and yields wrong results when a goal has not been detected correctly. The second method is more flexible and robust not relying on the correctness of single extracted features, but also is slower. In practice one tries to 39

42 apply the first method and if it fails, the second method is used Direct Localization Direct Localization is only applicable when both goals have been detected. Consider figure 3.17 for the following description. The idea is to determine the angle δ, because knowing Figure 3.17: Direct Localization. the distance of the blue goal, the robot s position and viewing direction can be calculated immediately. It is important to understand that δ is different from δ. This is because 40

43 the distance function is not linear. To obtain δ the vectors a and b that reach from the perception origin to the two goals first have to be mapped to local world coordinates. If a w and b w denote these mapped vectors, we define c w := a w b w. Now δ is the angle between c w and b w. Hence the position p of the robot lays on a line at an angle of δ to the line connecting the two goals. Knowing the distance to the blue goal determines p. The viewing direction also is known, because the angle α at which the yellow goal appears in the image in respect to the robot s viewing direction is preserved by the optical mapping. Of course, for constructing the position and viewing direction either of both goals can be used. In practice one will choose the nearer goal, since its position is more likely to be precise General Localization The direct localization method is only applicable when both goals have been found. Furthermore it relies on the correctness of the retrieved positions and does not use any other features as the walls for example. Another disadvantage is that the method is only applicable for a specific environment. It would be better to have a method that works with any environment. Therefore a second localization method has been developed, which uses a CAD-model for the specification of the environment. It consists of two steps. The first step determines some kind of plausibility for several positions where the robot could be. In a second step the positions are investigated in order of their plausibility, until the correct position has been found. In most cases only one or two positions have to be investigated. For determining the plausibility of each position the accumulation technique is applied. When a certain feature is detected in the image, the localization algorithm scans the CAD-model, finds all elements that have this feature and accumulates the clues for all possible positions of the robot indicated by each feature. For instance, if a yellow cluster has been detected in a certain distance, we have the clue that the robot must be on a circle around the yellow goal, since the yellow goal is the only yellow element in the CAD model. But if a wall is detected in the image, the algorithm finds four walls in the CAD model and takes account for each. The algorithm has similarities to the Markov Localization proposed by [7], but rather uses a grid to represent probabilities than particles. To accumulate indications we divide the playing field into m n cells. Each cell of the grid corresponds to a number that expresses the likelihood of the robot being on the cell. But the numbers are not limited to one but can have any value greater or equal to zero. One can imagine the grid to be a sheet of paper where different kinds of drawings can be made on, and that initially is blank (all cells are zero). The simplest entry one can make is a point, if there is an indication that the robot stands at a certain position. Next, there are lines and circles. Often the point, line or circle, where the robot is estimated to be on, are tainted with some uncertainty and instead of marking single cells, drawing thin lines or circles, one wants to specify some surrounding cells also. Since lines and circles can be composed by points, let s first consider points: The idea is to assign the greatest value to the cell where the point directly lays on, and to use lower numbers for the surrounding cells by using a certain distribution, for example a Gaussian one. It has to be emphasized, 41

44 that the cells do not specify real probabilities but just a value that specifies the strength of indications. For instance, if one wants to specify two positions with different strength of indication one can scale the Gaussian distributions differently. Hence the distributions are not real Gaussians but just have the shape of them. Not only the scale, but also the width of the distribution (the variance for real Gaussians) can be varied. For instance, if one has only a rough estimation of the robot s pose one can use a wide distribution. Figure 3.18 shows some possibilities. It is important to understand, that entries are made to the grid Figure 3.18: Points. Points can be specified using different Gaussian shapes. by adding values to the cells. Hence if several entries are near together they accumulate as shown in figure 3.19 where the accumulated bulges almost form a line. By decreasing the distance of the entries lines and circles can be approximated. Figure 3.19: Line. Points that are spaced at small intervals accumulate to a line. Suppose we know that the robot must be on a certain circle and also on a certain line. Then we can accumulate the line and the circle on the grid, and as a result of the accumulation, the intersections of them will have the greatest values. This is illustrated in figure The feature Distance of the goals If the distance to goal x is denoted by r x, then a semi-circle with radius r x is drawn around goal x. The circles will be drawn more fuzzy for great distances, since the estimation of r x becomes worse. This is illustrated in figure 3.21 for three different distances. Suppose, we can estimate the distance to the blue goal as well as to the yellow goal. Then we can draw two circles that might accumulate as shown in figure The intersections of the circles 42

45 Figure 3.20: Accumulation of a line and a circle. The intersections can be found by seeking for the points with maximal accumulation. Figure 3.21: Circles around the goals. If the distance of the robot to a specific goal is known, then the robot can be inferred to be on a circle around this goal. The greater the distance, the fuzzier the circles are drawn because the imprecision increases for greater distances. can be found by seeking for the cells with maximal accumulation. The feature Wall Unfortunately cases may occur, in which only one goal can be seen. This can happen for example when the distance to one goal is to great, due to optical problems with the mirror, or when the view is blocked. Therefore another feature has been used to indicate the robot s position: The best visible borderline. To detect this line we use the radial search, described above. Figure 3.23(a) gives an example, where the robot finds itself near a borderline. After the radial method has been used, and the retrieved transition points have been transformed into world coordinates, the Hough Transform is applied. Choosing the maximum activation in the parameter space, yields the best visible borderline, as depicted in figure 3.23(b). Now we know, that the robot must be located in a distance d from one of the borderlines. If we knew the border line that corresponded to the detected line, then we could draw a parallel line at the appropriate distance onto the accumulation grid. But not knowing the correspondence, we take account of all border lines, and draw parallel lines for them respectively. The resulting accumulation grid with the circles from the goals and the lines from the best visible wall -feature is shown in figure The positions that 43

46 Figure 3.22: Accumulation of two circles. The radii of the circles are the estimated distances of the robot to the blue and yellow goal. correspond to the maximal accumulations on the grid are marked in the left part of the figure. Extracting the positions of maximal accumulation After all entries have been made to the grid, the cell with maximal accumulation is searched for. Next, we calculate a certain error for that position, using a technique from tracking. Therefore the description of how to determine this error follows in the Tracking -section. For the moment it is sufficient to know that the error expresses the quality of how the model can be registered to the image, assuming that the robot stands on the specified position. Next, the second maximum will be extracted. But there arises a problem: As the maxima have an extend over several cells of the grid, it is very likely that the second maximum will just be at the neighboring cell. A possible method to cope with this problem is to zero all cells within a certain distance, after a maximum has been detected. In this manner all maxima can be retrieved in order of their accumulation values. This is illustrated in figure 3.25 for the extraction of four maxima. The extraction can be terminated if the error of registration is small enough or when a predefined number is reached. In the latter case the position with the smallest error is chosen. 44

47 (a) (b) Figure 3.23: Detecting the best visible line. (a) The transition to the walls are searched for. (b) The transition points are transformed to world coordinates, and the bestvisible line is detected using the Hough Transform. The robot (origin) is located in a distance d to this line. Flexibility The advantage of this general localization method is that when combined with a CADmodel, the environment can change without need to change the algorithm. For instance, if the playing field is changed to have walls that form a pentagon, this change has just to be specified in the CAD-model. After the algorithm has determined the distance to the best visible wall, it scans the CAD-model for walls and draws lines for all of them onto the accumulation grid. Consider the case that there are three goals, where two are yellow and one is blue. Specifying this change in the CAD-model would cause the algorithm to draw two circles when a yellow cluster has been found in the image. Although these cases will certainly not occur in RoboCup they will arise in different environments. 45

48 Figure 3.24: Result of accumulation. The circles stem from the goals and the lines from the best visible wall - feature. Figure 3.25: Extracting maxima. The maxima are extracted in the order of their accumulation values. After each extraction a spherical area is zeroed to ensure that consecutive maxima are spaced at a minimum distance. 46

49 Chapter 4 Tracking 4.1 Tracking the Playing Field Tracking the playing field starts after the robot has been localized successfully. A specific CAD-model is used for tracking. It is automatically constructed from the CAD-model specified by the user. According to figure 4.1 for each line in the original CAD-model, orthogonal equidistant track lines are added to the track model. The groups of all track lines that belong to the same line in the original model will be referred as track grids in the following. Figure 4.1: Track model. For each line in the user specified CAD-model a track grid is added automatically. Each track line is mapped into the image and the specified transitions are searched for. This is done by mapping the endpoints of each track line into the image and then reconnecting them with a line. This is imprecise because the mapped lines would be curves, but the error can be neglected. Points whose z-coordinate equals zero can be mapped by the inverse distance function, but the points of the track grids corresponding to the goalposts have positive z-coordinates. The following observation, illustrated in figure 4.2, helps to approximate their mapping: For a point p with a positive z-coordinate there exists a point p which lays on the same ray as p and is therefore mapped to the same position in the image. Since its z-coordinate equals zero the inverse distance function can be used for the mapping. To calculate p the ray r has to be determined. For a precise calculation point q 47

50 would have to be determined. But since only an approximation is needed the fixed point c (i.e. center of the mirror) can be used. Figure 4.2: Mapping points with positive z-coordinate. The point p should be mapped to the image. Point p lays on the same ray as p. Hence it is mapped to the same position in the image. Since p s z-coordinate equals zero the inverse distance function can be applied. To approximate the ray the fixed point c (center of the mirror) can be used instead of q, which simplifies the calculation. Figure 4.3 shows the mapped track grids. It can be seen that the model does not fit precisely to the playing field in the image. This is because initial robot localization was imprecise. But tracking will correct this error. Figure 4.4 shows the track model seen from above, with the found transitions marked by gray bullets. Track lines for which no transitions were found are not drawn. The key idea now is to calculate a 2-D rigid body transformation that brings the model in correspondence with the found transitions. In the case of figure 4.4, the model should be rotated to the left. To determine the rigid body transformation of the model, first a rotation and translation is calculated for each track grid independently. Next, the results are mixed together in a special manner to obtain a transformation for the whole model. There are two kinds of track grids that are processed differently: horizontal and vertical track grids. The horizontal grids track the walls of the playing field and the vertical grids track the goalposts Processing Horizontal Track Grids Figure 4.5 shows a single track grid, for which a rotation and a translation will be calculated. Let t 1, t 2,..., t n be the n transition points. Model line l is the line that passes through the centers of the track lines. It corresponds to a wall of the playing field, that will be referred as model line in the following. It is given by l : x = p + λ u (4.1) where p is the center of the first track line and u is the direction of l. The vector n is the vector that is orthogonal to u and is normalized to length 1. First, we calculate the vector r, the new proposed direction of the track grid. r = n 1 i=1 t i+1 t i t i+1 t i 48 (4.2)

51 Figure 4.3: Mapping the track model into the image. Since the robot has been localized successfully before tracking starts, the track model can be transformed to the robot s coordinate system and mapped to the image. The black bullets mark the found transitions. where denotes the Euclidean distance. The angle ϕ now can be calculated by ϕ = u r u r (4.3) Next, we calculate a translation vector v for the track grid, that points to the direction of the track lines and has a length that equals the average distance of the transition points to the model line. v = 1 n ( t i p) n (4.4) n i= Processing Vertical Track Grids For the vertical track grids only a rotation ϕ is calculated respectively. This is simply done by calculating the angle between the transition points and the center of the track grid, as depicted in figure Combining Displacements and Angles For all track grids a pair ( ϕ k, v k ) has been computed in the way described above, where k = 1,..., m, assuming m track grids. These pairs will be combined to yield a single pair ( ϕ g, v g ) that will determine the 2-D rigid body transformation for the CAD-model. 49

52 Figure 4.4: Horizontal track grids. The bullets on the lines mark the found transitions. Track lines for which no transition has been found are not drawn. Each track grid votes for a certain displacement and a certain angle. But the votes are not equally weighted. Track grids that have many found transitions will have a greater weight than track grids with few found transitions. If n k is the number of track lines in track grid k, and f k denotes the number of found transitions in this grid. Then the weight w k of track grid k is defined by: w k := f k n k (4.5) Then the translation of the model is calculated by the weighted sum: m v g = w k v k (4.6) k=1 The combination of angles is performed differently. This is because we don t want to mix angles that are too different. For instance, if two track grids propose an angle of 50 and one track grid proposes an angle of 50 and all track grids have the same weight, averaging would result in an angle of approximately But we want to take the 50 because for this angle there are the most votes. Hence, for combining the angles, the evidence aggregation method described above is used. Therefore the interval [0, ] is divided into 250 bins. The weight of the votes is w k, and the entries have the form of scaled Gaussians. The angle with maximal accumulation is chosen for ϕ g. 50

53 Figure 4.5: Calculation for a horizontal track grid. Single track grid, for which a rotation and a translation will be calculated Updating the Position of the CAD-model Next, the CAD-model, including the track grids, is rotated around its center and translated according to the pair ( ϕ k, v k ). Repeating the seek for transitions and updating the positions of the model while perceiving a sequence of images yields the desired result: the pose of the playing field seen from the robots point of view is tracked and so the position and the viewing direction of the robot on the playing field is known by a simple coordinate transformation. To reduce the variance of the estimated pose, a Kalmann filter can be used to combine the measurements from different frames. Figure 4.7(a) shows how the playing field is tracked while the robot rotates. 51

54 Figure 4.6: Vertical track grid. Here a single track grid is drawn isolated from the other grids to demonstrate how the angle ϕ is calculated Registration for Initial Search During initial search some different positions are verified to be the robot s position by making use of the tracking mechanism. Given a hypothetical position p of the robot on the playing field and an image where we can detect one of the two goals we can easily calculate the viewing direction α of the robot. Having the position and the viewing direction, we are able to transform the track grid in the way that it should match with the image, if the robot really is at the specified position. We can use the number of tracking lines with found transition as the base to define an error measure E pα for the registration: E pα := 1 1 m w i (4.7) m i=1 Here w i is the weight of track grid i as defined above, and m is the number of track grids. 4.2 Tracking Ball and Obstacles Once the ball and the obstacles have been found, a little rectangle is stretched around them respectively as illustrated in figure 4.7(b). It is anticipated, that the objects also will be within these rectangles in the next video image. Therefore the objects are only searched for in the specified regions. To find the ball within the rectangle exactly the same algorithm as for initial search is used. Even for the obstacles the same algorithm is applied, solely changing the color class that is looked for. If at some point an object cannot be found 52

55 (a) (b) Figure 4.7: Tracking of (a) the field while rotating; (b) ball and obstacles. within its rectangle, initial search is started again. For the obstacles, the positions of all objects are determined anew, when only one looses lock during tracking. To reduce the number of restarts, one can apply a motion model to predict the positions of the objects. The simplest one is to assign a velocity vector to each object that can be measured by the last observations of the positions. Here again a Kalmann filter can be applied. 53

56 Chapter 5 Implementation and Results 5.1 Implementation The software system has been implemented with Microsoft Visual C for Windows. The program divides into an image processing part, a CAD-part and a perceptional part which covers video input, initial search and tracking. Furthermore various diagrams and visualizations have been programmed. Figure 5.1 shows a screen shot of the program. Figure 5.1: Screen shot. 54

using an omnidirectional camera, sufficient information for controlled play can be collected. Another example for the use of omnidirectional cameras i

using an omnidirectional camera, sufficient information for controlled play can be collected. Another example for the use of omnidirectional cameras i An Omnidirectional Vision System that finds and tracks color edges and blobs Felix v. Hundelshausen, Sven Behnke, and Raul Rojas Freie Universität Berlin, Institut für Informatik Takustr. 9, 14195 Berlin,

More information

(a) (b) (c) Fig. 1. Omnidirectional camera: (a) principle; (b) physical construction; (c) captured. of a local vision system is more challenging than

(a) (b) (c) Fig. 1. Omnidirectional camera: (a) principle; (b) physical construction; (c) captured. of a local vision system is more challenging than An Omnidirectional Vision System that finds and tracks color edges and blobs Felix v. Hundelshausen, Sven Behnke, and Raul Rojas Freie Universität Berlin, Institut für Informatik Takustr. 9, 14195 Berlin,

More information

Lecture 9: Hough Transform and Thresholding base Segmentation

Lecture 9: Hough Transform and Thresholding base Segmentation #1 Lecture 9: Hough Transform and Thresholding base Segmentation Saad Bedros sbedros@umn.edu Hough Transform Robust method to find a shape in an image Shape can be described in parametric form A voting

More information

Horus: Object Orientation and Id without Additional Markers

Horus: Object Orientation and Id without Additional Markers Computer Science Department of The University of Auckland CITR at Tamaki Campus (http://www.citr.auckland.ac.nz) CITR-TR-74 November 2000 Horus: Object Orientation and Id without Additional Markers Jacky

More information

HOUGH TRANSFORM CS 6350 C V

HOUGH TRANSFORM CS 6350 C V HOUGH TRANSFORM CS 6350 C V HOUGH TRANSFORM The problem: Given a set of points in 2-D, find if a sub-set of these points, fall on a LINE. Hough Transform One powerful global method for detecting edges

More information

Calibrating an Overhead Video Camera

Calibrating an Overhead Video Camera Calibrating an Overhead Video Camera Raul Rojas Freie Universität Berlin, Takustraße 9, 495 Berlin, Germany http://www.fu-fighters.de Abstract. In this section we discuss how to calibrate an overhead video

More information

Light: Geometric Optics

Light: Geometric Optics Light: Geometric Optics Regular and Diffuse Reflection Sections 23-1 to 23-2. How We See Weseebecauselightreachesoureyes. There are two ways, therefore, in which we see: (1) light from a luminous object

More information

Canny Edge Based Self-localization of a RoboCup Middle-sized League Robot

Canny Edge Based Self-localization of a RoboCup Middle-sized League Robot Canny Edge Based Self-localization of a RoboCup Middle-sized League Robot Yoichi Nakaguro Sirindhorn International Institute of Technology, Thammasat University P.O. Box 22, Thammasat-Rangsit Post Office,

More information

Using Layered Color Precision for a Self-Calibrating Vision System

Using Layered Color Precision for a Self-Calibrating Vision System ROBOCUP2004 SYMPOSIUM, Instituto Superior Técnico, Lisboa, Portugal, July 4-5, 2004. Using Layered Color Precision for a Self-Calibrating Vision System Matthias Jüngel Institut für Informatik, LFG Künstliche

More information

Robust Color Choice for Small-size League RoboCup Competition

Robust Color Choice for Small-size League RoboCup Competition Robust Color Choice for Small-size League RoboCup Competition Qiang Zhou Limin Ma David Chelberg David Parrott School of Electrical Engineering and Computer Science, Ohio University Athens, OH 45701, U.S.A.

More information

Particle-Filter-Based Self-Localization Using Landmarks and Directed Lines

Particle-Filter-Based Self-Localization Using Landmarks and Directed Lines Particle-Filter-Based Self-Localization Using Landmarks and Directed Lines Thomas Röfer 1, Tim Laue 1, and Dirk Thomas 2 1 Center for Computing Technology (TZI), FB 3, Universität Bremen roefer@tzi.de,

More information

An Efficient Need-Based Vision System in Variable Illumination Environment of Middle Size RoboCup

An Efficient Need-Based Vision System in Variable Illumination Environment of Middle Size RoboCup An Efficient Need-Based Vision System in Variable Illumination Environment of Middle Size RoboCup Mansour Jamzad and Abolfazal Keighobadi Lamjiri Sharif University of Technology Department of Computer

More information

4. A bulb has a luminous flux of 2400 lm. What is the luminous intensity of the bulb?

4. A bulb has a luminous flux of 2400 lm. What is the luminous intensity of the bulb? 1. Match the physical quantities (first column) with the units (second column). 4. A bulb has a luminous flux of 2400 lm. What is the luminous intensity of the bulb? (π=3.) Luminous flux A. candela Radiant

More information

Color and Shading. Color. Shapiro and Stockman, Chapter 6. Color and Machine Vision. Color and Perception

Color and Shading. Color. Shapiro and Stockman, Chapter 6. Color and Machine Vision. Color and Perception Color and Shading Color Shapiro and Stockman, Chapter 6 Color is an important factor for for human perception for object and material identification, even time of day. Color perception depends upon both

More information

Chapter 4. Clustering Core Atoms by Location

Chapter 4. Clustering Core Atoms by Location Chapter 4. Clustering Core Atoms by Location In this chapter, a process for sampling core atoms in space is developed, so that the analytic techniques in section 3C can be applied to local collections

More information

Chapter 32 Light: Reflection and Refraction. Copyright 2009 Pearson Education, Inc.

Chapter 32 Light: Reflection and Refraction. Copyright 2009 Pearson Education, Inc. Chapter 32 Light: Reflection and Refraction Units of Chapter 32 The Ray Model of Light Reflection; Image Formation by a Plane Mirror Formation of Images by Spherical Mirrors Index of Refraction Refraction:

More information

Physics for Scientists & Engineers 2

Physics for Scientists & Engineers 2 Geometric Optics Physics for Scientists & Engineers 2 Spring Semester 2005 Lecture 36! The study of light divides itself into three fields geometric optics wave optics quantum optics! In the previous chapter,

More information

EXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006,

EXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, School of Computer Science and Communication, KTH Danica Kragic EXAM SOLUTIONS Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, 14.00 19.00 Grade table 0-25 U 26-35 3 36-45

More information

form are graphed in Cartesian coordinates, and are graphed in Cartesian coordinates.

form are graphed in Cartesian coordinates, and are graphed in Cartesian coordinates. Plot 3D Introduction Plot 3D graphs objects in three dimensions. It has five basic modes: 1. Cartesian mode, where surfaces defined by equations of the form are graphed in Cartesian coordinates, 2. cylindrical

More information

Improving Vision-Based Distance Measurements using Reference Objects

Improving Vision-Based Distance Measurements using Reference Objects Improving Vision-Based Distance Measurements using Reference Objects Matthias Jüngel, Heinrich Mellmann, and Michael Spranger Humboldt-Universität zu Berlin, Künstliche Intelligenz Unter den Linden 6,

More information

Part Images Formed by Flat Mirrors. This Chapter. Phys. 281B Geometric Optics. Chapter 2 : Image Formation. Chapter 2: Image Formation

Part Images Formed by Flat Mirrors. This Chapter. Phys. 281B Geometric Optics. Chapter 2 : Image Formation. Chapter 2: Image Formation Phys. 281B Geometric Optics This Chapter 3 Physics Department Yarmouk University 21163 Irbid Jordan 1- Images Formed by Flat Mirrors 2- Images Formed by Spherical Mirrors 3- Images Formed by Refraction

More information

Self-calibration of a pair of stereo cameras in general position

Self-calibration of a pair of stereo cameras in general position Self-calibration of a pair of stereo cameras in general position Raúl Rojas Institut für Informatik Freie Universität Berlin Takustr. 9, 14195 Berlin, Germany Abstract. This paper shows that it is possible

More information

TEAMS National Competition Middle School Version Photometry Solution Manual 25 Questions

TEAMS National Competition Middle School Version Photometry Solution Manual 25 Questions TEAMS National Competition Middle School Version Photometry Solution Manual 25 Questions Page 1 of 14 Photometry Questions 1. When an upright object is placed between the focal point of a lens and a converging

More information

GEOMETRIC OPTICS MIRRORS

GEOMETRIC OPTICS MIRRORS GEOMETRIC OPTICS Now that we understand the laws of reflection and refraction we can put them to practical use by designing optical instruments. We begin with the law of reflection which tells us that

More information

Towards a Calibration-Free Robot: The ACT Algorithm for Automatic Online Color Training

Towards a Calibration-Free Robot: The ACT Algorithm for Automatic Online Color Training Towards a Calibration-Free Robot: The ACT Algorithm for Automatic Online Color Training Patrick Heinemann, Frank Sehnke, Felix Streichert, and Andreas Zell Wilhelm-Schickard-Institute, Department of Computer

More information

Introduction. Chapter Overview

Introduction. Chapter Overview Chapter 1 Introduction The Hough Transform is an algorithm presented by Paul Hough in 1962 for the detection of features of a particular shape like lines or circles in digitalized images. In its classical

More information

IRIS SEGMENTATION OF NON-IDEAL IMAGES

IRIS SEGMENTATION OF NON-IDEAL IMAGES IRIS SEGMENTATION OF NON-IDEAL IMAGES William S. Weld St. Lawrence University Computer Science Department Canton, NY 13617 Xiaojun Qi, Ph.D Utah State University Computer Science Department Logan, UT 84322

More information

2D rendering takes a photo of the 2D scene with a virtual camera that selects an axis aligned rectangle from the scene. The photograph is placed into

2D rendering takes a photo of the 2D scene with a virtual camera that selects an axis aligned rectangle from the scene. The photograph is placed into 2D rendering takes a photo of the 2D scene with a virtual camera that selects an axis aligned rectangle from the scene. The photograph is placed into the viewport of the current application window. A pixel

More information

CS 231A Computer Vision (Winter 2014) Problem Set 3

CS 231A Computer Vision (Winter 2014) Problem Set 3 CS 231A Computer Vision (Winter 2014) Problem Set 3 Due: Feb. 18 th, 2015 (11:59pm) 1 Single Object Recognition Via SIFT (45 points) In his 2004 SIFT paper, David Lowe demonstrates impressive object recognition

More information

CIRCULAR MOIRÉ PATTERNS IN 3D COMPUTER VISION APPLICATIONS

CIRCULAR MOIRÉ PATTERNS IN 3D COMPUTER VISION APPLICATIONS CIRCULAR MOIRÉ PATTERNS IN 3D COMPUTER VISION APPLICATIONS Setiawan Hadi Mathematics Department, Universitas Padjadjaran e-mail : shadi@unpad.ac.id Abstract Geometric patterns generated by superimposing

More information

Lecture 15: Segmentation (Edge Based, Hough Transform)

Lecture 15: Segmentation (Edge Based, Hough Transform) Lecture 15: Segmentation (Edge Based, Hough Transform) c Bryan S. Morse, Brigham Young University, 1998 000 Last modified on February 3, 000 at :00 PM Contents 15.1 Introduction..............................................

More information

TEAMS National Competition High School Version Photometry Solution Manual 25 Questions

TEAMS National Competition High School Version Photometry Solution Manual 25 Questions TEAMS National Competition High School Version Photometry Solution Manual 25 Questions Page 1 of 15 Photometry Questions 1. When an upright object is placed between the focal point of a lens and a converging

More information

AUTOMATED 4 AXIS ADAYfIVE SCANNING WITH THE DIGIBOTICS LASER DIGITIZER

AUTOMATED 4 AXIS ADAYfIVE SCANNING WITH THE DIGIBOTICS LASER DIGITIZER AUTOMATED 4 AXIS ADAYfIVE SCANNING WITH THE DIGIBOTICS LASER DIGITIZER INTRODUCTION The DIGIBOT 3D Laser Digitizer is a high performance 3D input device which combines laser ranging technology, personal

More information

Last update: May 4, Vision. CMSC 421: Chapter 24. CMSC 421: Chapter 24 1

Last update: May 4, Vision. CMSC 421: Chapter 24. CMSC 421: Chapter 24 1 Last update: May 4, 200 Vision CMSC 42: Chapter 24 CMSC 42: Chapter 24 Outline Perception generally Image formation Early vision 2D D Object recognition CMSC 42: Chapter 24 2 Perception generally Stimulus

More information

Real time game field limits recognition for robot self-localization using collinearity in Middle-Size RoboCup Soccer

Real time game field limits recognition for robot self-localization using collinearity in Middle-Size RoboCup Soccer Real time game field limits recognition for robot self-localization using collinearity in Middle-Size RoboCup Soccer Fernando Ribeiro (1) Gil Lopes (2) (1) Department of Industrial Electronics, Guimarães,

More information

3D object recognition used by team robotto

3D object recognition used by team robotto 3D object recognition used by team robotto Workshop Juliane Hoebel February 1, 2016 Faculty of Computer Science, Otto-von-Guericke University Magdeburg Content 1. Introduction 2. Depth sensor 3. 3D object

More information

Lecture Outlines Chapter 26

Lecture Outlines Chapter 26 Lecture Outlines Chapter 26 11/18/2013 2 Chapter 26 Geometrical Optics Objectives: After completing this module, you should be able to: Explain and discuss with diagrams, reflection and refraction of light

More information

Physics 11. Unit 8 Geometric Optics Part 1

Physics 11. Unit 8 Geometric Optics Part 1 Physics 11 Unit 8 Geometric Optics Part 1 1.Review of waves In the previous section, we have investigated the nature and behaviors of waves in general. We know that all waves possess the following characteristics:

More information

4) Finish the spline here. To complete the spline, double click the last point or select the spline tool again.

4) Finish the spline here. To complete the spline, double click the last point or select the spline tool again. 1) Select the line tool 3) Move the cursor along the X direction (be careful to stay on the X axis alignment so that the line is perpendicular) and click for the second point of the line. Type 0.5 for

More information

f. (5.3.1) So, the higher frequency means the lower wavelength. Visible part of light spectrum covers the range of wavelengths from

f. (5.3.1) So, the higher frequency means the lower wavelength. Visible part of light spectrum covers the range of wavelengths from Lecture 5-3 Interference and Diffraction of EM Waves During our previous lectures we have been talking about electromagnetic (EM) waves. As we know, harmonic waves of any type represent periodic process

More information

34.2: Two Types of Image

34.2: Two Types of Image Chapter 34 Images 34.2: Two Types of Image For you to see an object, your eye intercepts some of the light rays spreading from the object and then redirect them onto the retina at the rear of the eye.

More information

Camera model and multiple view geometry

Camera model and multiple view geometry Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then

More information

Chapter 26 Geometrical Optics

Chapter 26 Geometrical Optics Chapter 26 Geometrical Optics 1 Overview of Chapter 26 The Reflection of Light Forming Images with a Plane Mirror Spherical Mirrors Ray Tracing and the Mirror Equation The Refraction of Light Ray Tracing

More information

Phys102 Lecture 21/22 Light: Reflection and Refraction

Phys102 Lecture 21/22 Light: Reflection and Refraction Phys102 Lecture 21/22 Light: Reflection and Refraction Key Points The Ray Model of Light Reflection and Mirrors Refraction, Snell s Law Total internal Reflection References 23-1,2,3,4,5,6. The Ray Model

More information

Computer Vision I - Appearance-based Matching and Projective Geometry

Computer Vision I - Appearance-based Matching and Projective Geometry Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation

More information

Education and Training CUFMEM14A. Exercise 2. Create, Manipulate and Incorporate 2D Graphics

Education and Training CUFMEM14A. Exercise 2. Create, Manipulate and Incorporate 2D Graphics Education and Training CUFMEM14A Exercise 2 Create, Manipulate and Incorporate 2D Graphics Menu Exercise 2 Exercise 2a: Scarecrow Exercise - Painting and Drawing Tools... 3 Exercise 2b: Scarecrow Exercise

More information

Chapter 8: Physical Optics

Chapter 8: Physical Optics Chapter 8: Physical Optics Whether light is a particle or a wave had puzzled physicists for centuries. In this chapter, we only analyze light as a wave using basic optical concepts such as interference

More information

Issues with Curve Detection Grouping (e.g., the Canny hysteresis thresholding procedure) Model tting They can be performed sequentially or simultaneou

Issues with Curve Detection Grouping (e.g., the Canny hysteresis thresholding procedure) Model tting They can be performed sequentially or simultaneou an edge image, nd line or curve segments present Given the image. in Line and Curves Detection 1 Issues with Curve Detection Grouping (e.g., the Canny hysteresis thresholding procedure) Model tting They

More information

Chapter 23. Geometrical Optics (lecture 1: mirrors) Dr. Armen Kocharian

Chapter 23. Geometrical Optics (lecture 1: mirrors) Dr. Armen Kocharian Chapter 23 Geometrical Optics (lecture 1: mirrors) Dr. Armen Kocharian Reflection and Refraction at a Plane Surface The light radiate from a point object in all directions The light reflected from a plane

More information

CS4758: Rovio Augmented Vision Mapping Project

CS4758: Rovio Augmented Vision Mapping Project CS4758: Rovio Augmented Vision Mapping Project Sam Fladung, James Mwaura Abstract The goal of this project is to use the Rovio to create a 2D map of its environment using a camera and a fixed laser pointer

More information

(Refer Slide Time 00:17) Welcome to the course on Digital Image Processing. (Refer Slide Time 00:22)

(Refer Slide Time 00:17) Welcome to the course on Digital Image Processing. (Refer Slide Time 00:22) Digital Image Processing Prof. P. K. Biswas Department of Electronics and Electrical Communications Engineering Indian Institute of Technology, Kharagpur Module Number 01 Lecture Number 02 Application

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based

More information

PHY 222 Lab 11 Interference and Diffraction Patterns Investigating interference and diffraction of light waves

PHY 222 Lab 11 Interference and Diffraction Patterns Investigating interference and diffraction of light waves PHY 222 Lab 11 Interference and Diffraction Patterns Investigating interference and diffraction of light waves Print Your Name Print Your Partners' Names Instructions April 17, 2015 Before lab, read the

More information

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection CHAPTER 3 Single-view Geometry When we open an eye or take a photograph, we see only a flattened, two-dimensional projection of the physical underlying scene. The consequences are numerous and startling.

More information

Chapter 26 Geometrical Optics

Chapter 26 Geometrical Optics Chapter 26 Geometrical Optics 26.1 The Reflection of Light 26.2 Forming Images With a Plane Mirror 26.3 Spherical Mirrors 26.4 Ray Tracing and the Mirror Equation 26.5 The Refraction of Light 26.6 Ray

More information

Robust and Accurate Detection of Object Orientation and ID without Color Segmentation

Robust and Accurate Detection of Object Orientation and ID without Color Segmentation 0 Robust and Accurate Detection of Object Orientation and ID without Color Segmentation Hironobu Fujiyoshi, Tomoyuki Nagahashi and Shoichi Shimizu Chubu University Japan Open Access Database www.i-techonline.com

More information

Shadows in the graphics pipeline

Shadows in the graphics pipeline Shadows in the graphics pipeline Steve Marschner Cornell University CS 569 Spring 2008, 19 February There are a number of visual cues that help let the viewer know about the 3D relationships between objects

More information

3. Confirm Does the law of reflection apply to rough surfaces? Explain. Diffuse Reflection

3. Confirm Does the law of reflection apply to rough surfaces? Explain. Diffuse Reflection Light Key Concepts How does light reflect from smooth surfaces and rough surfaces? What happens to light when it strikes a concave mirror? Which types of mirrors can produce a virtual image? Reflection

More information

Real-Time Color Coded Object Detection Using a Modular Computer Vision Library

Real-Time Color Coded Object Detection Using a Modular Computer Vision Library Real-Time Color Coded Object Detection Using a Modular Computer Vision Library Alina Trifan 1, António J. R. Neves 2, Bernardo Cunha 3 and José Luís Azevedo 4 1 IEETA/ DETI, University of Aveiro Aveiro,

More information

Planes Intersecting Cones: Static Hypertext Version

Planes Intersecting Cones: Static Hypertext Version Page 1 of 12 Planes Intersecting Cones: Static Hypertext Version On this page, we develop some of the details of the plane-slicing-cone picture discussed in the introduction. The relationship between the

More information

Digital Image Correlation of Stereoscopic Images for Radial Metrology

Digital Image Correlation of Stereoscopic Images for Radial Metrology Digital Image Correlation of Stereoscopic Images for Radial Metrology John A. Gilbert Professor of Mechanical Engineering University of Alabama in Huntsville Huntsville, AL 35899 Donald R. Matthys Professor

More information

Types of Edges. Why Edge Detection? Types of Edges. Edge Detection. Gradient. Edge Detection

Types of Edges. Why Edge Detection? Types of Edges. Edge Detection. Gradient. Edge Detection Why Edge Detection? How can an algorithm extract relevant information from an image that is enables the algorithm to recognize objects? The most important information for the interpretation of an image

More information

The exam begins at 2:40pm and ends at 4:00pm. You must turn your exam in when time is announced or risk not having it accepted.

The exam begins at 2:40pm and ends at 4:00pm. You must turn your exam in when time is announced or risk not having it accepted. CS 184: Foundations of Computer Graphics page 1 of 12 Student Name: Student ID: Instructions: Read them carefully! The exam begins at 2:40pm and ends at 4:00pm. You must turn your exam in when time is

More information

Chapter 23. Geometrical Optics: Mirrors and Lenses and other Instruments

Chapter 23. Geometrical Optics: Mirrors and Lenses and other Instruments Chapter 23 Geometrical Optics: Mirrors and Lenses and other Instruments HITT1 A small underwater pool light is 1 m below the surface of a swimming pool. What is the radius of the circle of light on the

More information

Chapter 12 Notes: Optics

Chapter 12 Notes: Optics Chapter 12 Notes: Optics How can the paths traveled by light rays be rearranged in order to form images? In this chapter we will consider just one form of electromagnetic wave: visible light. We will be

More information

Robotics Programming Laboratory

Robotics Programming Laboratory Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car

More information

Conceptual Physics 11 th Edition

Conceptual Physics 11 th Edition Conceptual Physics 11 th Edition Chapter 28: REFLECTION & REFRACTION This lecture will help you understand: Reflection Principle of Least Time Law of Reflection Refraction Cause of Refraction Dispersion

More information

Sensor Modalities. Sensor modality: Different modalities:

Sensor Modalities. Sensor modality: Different modalities: Sensor Modalities Sensor modality: Sensors which measure same form of energy and process it in similar ways Modality refers to the raw input used by the sensors Different modalities: Sound Pressure Temperature

More information

This blog addresses the question: how do we determine the intersection of two circles in the Cartesian plane?

This blog addresses the question: how do we determine the intersection of two circles in the Cartesian plane? Intersecting Circles This blog addresses the question: how do we determine the intersection of two circles in the Cartesian plane? This is a problem that a programmer might have to solve, for example,

More information

specular diffuse reflection.

specular diffuse reflection. Lesson 8 Light and Optics The Nature of Light Properties of Light: Reflection Refraction Interference Diffraction Polarization Dispersion and Prisms Total Internal Reflection Huygens s Principle The Nature

More information

Computer Vision. Image Segmentation. 10. Segmentation. Computer Engineering, Sejong University. Dongil Han

Computer Vision. Image Segmentation. 10. Segmentation. Computer Engineering, Sejong University. Dongil Han Computer Vision 10. Segmentation Computer Engineering, Sejong University Dongil Han Image Segmentation Image segmentation Subdivides an image into its constituent regions or objects - After an image has

More information

Nelson Functions 11 Errata

Nelson Functions 11 Errata Nelson Functions 11 Errata 1: Introduction to Functions Location Question Correct Answer Getting Started 6a Graph is correct but vertex and axis of symmetry are not labelled. Add blue point labelled (in

More information

Computer Science 474 Spring 2010 Viewing Transformation

Computer Science 474 Spring 2010 Viewing Transformation Viewing Transformation Previous readings have described how to transform objects from one position and orientation to another. One application for such transformations is to create complex models from

More information

Nao Devils Dortmund. Team Description Paper for RoboCup Matthias Hofmann, Ingmar Schwarz, and Oliver Urbann

Nao Devils Dortmund. Team Description Paper for RoboCup Matthias Hofmann, Ingmar Schwarz, and Oliver Urbann Nao Devils Dortmund Team Description Paper for RoboCup 2017 Matthias Hofmann, Ingmar Schwarz, and Oliver Urbann Robotics Research Institute Section Information Technology TU Dortmund University 44221 Dortmund,

More information

What is it? How does it work? How do we use it?

What is it? How does it work? How do we use it? What is it? How does it work? How do we use it? Dual Nature http://www.youtube.com/watch?v=dfpeprq7ogc o Electromagnetic Waves display wave behavior o Created by oscillating electric and magnetic fields

More information

OBJECT detection in general has many applications

OBJECT detection in general has many applications 1 Implementing Rectangle Detection using Windowed Hough Transform Akhil Singh, Music Engineering, University of Miami Abstract This paper implements Jung and Schramm s method to use Hough Transform for

More information

Practice Exam Sample Solutions

Practice Exam Sample Solutions CS 675 Computer Vision Instructor: Marc Pomplun Practice Exam Sample Solutions Note that in the actual exam, no calculators, no books, and no notes allowed. Question 1: out of points Question 2: out of

More information

Tutorial on Range and Bearing Estimation

Tutorial on Range and Bearing Estimation Tutorial on Range and Bearing Estimation Basia Korel (bkorel@cs.brown.edu) October 30, 2009 1 Introduction Estimating the range and bearing from the robot to a landmark object is a necessary component

More information

Problem A. Interactive Smiley Face

Problem A. Interactive Smiley Face Problem A. Interactive Smiley Face 1 second Igor likes smiley faces a lot. He wrote a program that generates really large pictures of white and black pixels with smiley faces. Depending on Igor s mood,

More information

Physics 1C Lecture 26A. Beginning of Chapter 26

Physics 1C Lecture 26A. Beginning of Chapter 26 Physics 1C Lecture 26A Beginning of Chapter 26 Mirrors and Lenses! As we have noted before, light rays can be diverted by optical systems to fool your eye into thinking an object is somewhere that it is

More information

Massachusetts Institute of Technology. Department of Computer Science and Electrical Engineering /6.866 Machine Vision Quiz I

Massachusetts Institute of Technology. Department of Computer Science and Electrical Engineering /6.866 Machine Vision Quiz I Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision Quiz I Handed out: 2004 Oct. 21st Due on: 2003 Oct. 28th Problem 1: Uniform reflecting

More information

Computer Vision 6 Segmentation by Fitting

Computer Vision 6 Segmentation by Fitting Computer Vision 6 Segmentation by Fitting MAP-I Doctoral Programme Miguel Tavares Coimbra Outline The Hough Transform Fitting Lines Fitting Curves Fitting as a Probabilistic Inference Problem Acknowledgements:

More information

MATRIX: A force field pattern matching method for mobile robots

MATRIX: A force field pattern matching method for mobile robots MATRIX: A force field pattern matching method for mobile robots Felix von Hundelshausen, Michael Schreiber, Fabian Wiesel, Achim Liers, and Raúl Rojas Technical Report B-8-3 Freie Universität Berlin, Takustraße

More information

Feature descriptors and matching

Feature descriptors and matching Feature descriptors and matching Detections at multiple scales Invariance of MOPS Intensity Scale Rotation Color and Lighting Out-of-plane rotation Out-of-plane rotation Better representation than color:

More information

TEAM 12: TERMANATOR PROJECT PROPOSAL. TEAM MEMBERS: Donald Eng Rodrigo Ipince Kevin Luu

TEAM 12: TERMANATOR PROJECT PROPOSAL. TEAM MEMBERS: Donald Eng Rodrigo Ipince Kevin Luu TEAM 12: TERMANATOR PROJECT PROPOSAL TEAM MEMBERS: Donald Eng Rodrigo Ipince Kevin Luu 1. INTRODUCTION: This project involves the design and implementation of a unique, first-person shooting game. The

More information

COMP Preliminaries Jan. 6, 2015

COMP Preliminaries Jan. 6, 2015 Lecture 1 Computer graphics, broadly defined, is a set of methods for using computers to create and manipulate images. There are many applications of computer graphics including entertainment (games, cinema,

More information

Chapter 5. Projections and Rendering

Chapter 5. Projections and Rendering Chapter 5 Projections and Rendering Topics: Perspective Projections The rendering pipeline In order to view manipulate and view a graphics object we must find ways of storing it a computer-compatible way.

More information

4. Refraction. glass, air, Perspex and water.

4. Refraction. glass, air, Perspex and water. Mr. C. Grima 11 1. Rays and Beams A ray of light is a narrow beam of parallel light, which can be represented by a line with an arrow on it, in diagrams. A group of rays makes up a beam of light. In laboratory

More information

A Survey of Light Source Detection Methods

A Survey of Light Source Detection Methods A Survey of Light Source Detection Methods Nathan Funk University of Alberta Mini-Project for CMPUT 603 November 30, 2003 Abstract This paper provides an overview of the most prominent techniques for light

More information

Optics II. Reflection and Mirrors

Optics II. Reflection and Mirrors Optics II Reflection and Mirrors Geometric Optics Using a Ray Approximation Light travels in a straight-line path in a homogeneous medium until it encounters a boundary between two different media The

More information

S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T

S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T Copyright 2018 Sung-eui Yoon, KAIST freely available on the internet http://sglab.kaist.ac.kr/~sungeui/render

More information

Lecture Outline Chapter 26. Physics, 4 th Edition James S. Walker. Copyright 2010 Pearson Education, Inc.

Lecture Outline Chapter 26. Physics, 4 th Edition James S. Walker. Copyright 2010 Pearson Education, Inc. Lecture Outline Chapter 26 Physics, 4 th Edition James S. Walker Chapter 26 Geometrical Optics Units of Chapter 26 The Reflection of Light Forming Images with a Plane Mirror Spherical Mirrors Ray Tracing

More information

2/26/2016. Chapter 23 Ray Optics. Chapter 23 Preview. Chapter 23 Preview

2/26/2016. Chapter 23 Ray Optics. Chapter 23 Preview. Chapter 23 Preview Chapter 23 Ray Optics Chapter Goal: To understand and apply the ray model of light. Slide 23-2 Chapter 23 Preview Slide 23-3 Chapter 23 Preview Slide 23-4 1 Chapter 23 Preview Slide 23-5 Chapter 23 Preview

More information

Image Formation. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania

Image Formation. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania Image Formation Antonino Furnari Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania furnari@dmi.unict.it 18/03/2014 Outline Introduction; Geometric Primitives

More information

Character Recognition

Character Recognition Character Recognition 5.1 INTRODUCTION Recognition is one of the important steps in image processing. There are different methods such as Histogram method, Hough transformation, Neural computing approaches

More information

(Refer Slide Time: 0:32)

(Refer Slide Time: 0:32) Digital Image Processing. Professor P. K. Biswas. Department of Electronics and Electrical Communication Engineering. Indian Institute of Technology, Kharagpur. Lecture-57. Image Segmentation: Global Processing

More information

cse 252c Fall 2004 Project Report: A Model of Perpendicular Texture for Determining Surface Geometry

cse 252c Fall 2004 Project Report: A Model of Perpendicular Texture for Determining Surface Geometry cse 252c Fall 2004 Project Report: A Model of Perpendicular Texture for Determining Surface Geometry Steven Scher December 2, 2004 Steven Scher SteveScher@alumni.princeton.edu Abstract Three-dimensional

More information

Visual Recognition: Image Formation

Visual Recognition: Image Formation Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know

More information

Precise Omnidirectional Camera Calibration

Precise Omnidirectional Camera Calibration Precise Omnidirectional Camera Calibration Dennis Strelow, Jeffrey Mishler, David Koes, and Sanjiv Singh Carnegie Mellon University {dstrelow, jmishler, dkoes, ssingh}@cs.cmu.edu Abstract Recent omnidirectional

More information

Computer Vision 2. SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung. Computer Vision 2 Dr. Benjamin Guthier

Computer Vision 2. SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung. Computer Vision 2 Dr. Benjamin Guthier Computer Vision 2 SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung Computer Vision 2 Dr. Benjamin Guthier 1. IMAGE PROCESSING Computer Vision 2 Dr. Benjamin Guthier Content of this Chapter Non-linear

More information