SWARMING PIXEL TRACKER

Transcription

1 SWARMING PIXEL TRACKER B.Tech Project submitted by Parth Kanungo Computers and Communication Engineering, The LNM Institute of Information Technology, Jaipur 2011

2 CERTIFICATE It is certified that the work contained in the B.Tech. Project entitled Swarming Pixel Tracker by Parth Kanungo (Y07uc078) has been carried out under our supervision and that this work has not been submitted elsewhere for a degree. May, 2011 Soumitra Debnath, Assistant Professor, Communication & Computer Engg. The LNMIIT, Jaipur Prithwijit Guha, Visiting Faculty, Communication & Computer Engg. The LNMIIT, Jaipur

3 ABSTRACT Tracking moving objects is a critical task in computer vision, with many practical applications such as vision based interface tasks, visual surveillance or perceptual intelligence applications. This project presents a method of visual tracking using a swarm intelligence metaphor. Swarm Intelligence refers to the collective intelligent emergent behaviour of simpler agents. This is a concept inspired by the social behaviour of insects. Each agent (or swarm particle) is "unintelligent" and follows a set of primitive rules like "stay close to a neighbour". The mutual interactions and movements make the entire system "intelligent" and robust for various applications. For object tracking, we have assigned a tracker (i.e. a swarm particle) to each pixel of the object read in the first frame. As the subsequent frames of video come by, the motion of the trackers (or the swarm particles) is governed by their neighbours and their respective confidence values. The interactions of individual particles results in an emergent object tracking behaviour.

4 ACKNOWLEDGEMENT I would like to express my sincere gratitude to Prof. Prithwijit Guha for guiding and supporting me throughout the duration of this project. I would also like to thank my friend Dushyant Goyal for valuable discussions.

5 CONTENTS 1. Introduction to Swarms a. Introduction b. Mathematical Modelling of Swarm 2. Swarm Simulation a. The Data Structure b. Initialize a swarm c. Step response of a swarm d. Assigning goals to the swarm e. Behaviour in a two-tank system 3. Application in Visual Tracking a. Introduction b. Mathematical Model c. Results 4. Conclusions and Future Work 5. References

6 Chapter 1 Introduction to Swarms Flocking birds 1.1 Introduction Birds flock, ants forage, fish school. Evolution of creatures has produced swarming in so-many contexts. Swarming has many individual and group benefits. For example swarming provides ability to forage more effectively. In case of birds, it reduces the overall air drag on the birds giving them an ability to travel greater distances. Then there is a concept of safety associated as well. Fishes in swarms are safer compared with when they are alone. The dynamics of predatorprey algorithm are such that the predator gets confused when it sees multiple preys at the same time, and is unable to attack. This concept of swarms can be applied in computers to solve problems. In fact, many problems are ideal candidates for swarm algorithms, including traveling salesman problems, data fusion on distributed sensor networks, network load balancing, and the coordination of large numbers of machines, e.g., cars on the highway or automatons in a factory.

7 Before going into the details, let us establish a concrete definition of a swarm and an autonomous agent. An autonomous agent is an entity that is able to sense and affect its environment. It interacts with its environment, which probably consists of other agents, but acts relatively independently from all the other agents. It does not follow commands from a leader, or some global plan. Such agent is referred to as a swarm-particle (or individual) and the collection of these particles, which produces an emergent behaviour, is referred to as a swarm. Using the swarms in computer science, we employ what is referred to as swarm intelligence, a term first used by Beni, Hackwood, and Wang in regards to the self-organizing behaviours of cellular robotic systems [1]. This interpretation of swarm intelligence focuses on unintelligent agents with limited processing capabilities, but possessing behaviours that collectively are intelligent. This behaviour can be applied in optimization in telecommunicate systems, robotics, tracking patterns in transportation systems, and military applications. Thus, Computer Scientists have used the concept of swarm to formulate various algorithms like Particle Swarm Optimization and Ant colony Optimization. Then, there are BOIDs, which are simulated flocking creatures created by Craig Reynold. They have a lot of advantages in computer science they are robust, simple agents, distributed & parallel and effort magnification and scalability emerges in their behaviour. [2] In this project, instead of using these standard algorithms, we follow a more primitive approach of implementing basic motion rules to the swarm particles, as a result of which we get emergent swarm behaviour. 1.2 Mathematical Modelling of Swarm Swarm is an emergent behaviour arising from simple rules that are followed by particles (or individuals) and does not involve any central coordination. The simplest mathematical models of swarm follow the following rules: 1. Move in the same direction as your neighbours 2. Remain close to your neighbours 3. Avoid collisions with your neighbours This can be made possible by implementing either of the two models as shown in the figure:

8 The Metric Distance Model consists of three zones around an animal. In the zone of repulsion, closest to the animal, the focal animal will seek to distance itself from its neighbours to avoid collision. In the other zone zone of alignment, the particles will move in the direction of the neighbour. In the outermost zone, the animals attract each other, so as to make the swarm cohesive. In the other model known as the Topological Distance Model, the animal pays attention to only 6 or 7 animals closest to itself, regardless of the distance. To end this section, I would like to quote some lines from the novel PREY written by the Science Fiction author Micheal Chricton: In the last few years, artificial life had replaced artificial intelligence as a long-term computing goal. The idea was to write programs that had the attributes of living creatures the ability to adapt, cooperate, learn, adjust to change. Many of those qualities were especially important in robotics, and they were starting to be realized with distributed processing. Distributed processing meant that you divided your work among several processors, or among a network of virtual agents that you created in the computer. There were several basic ways this was done. One way was to create a large population of fairly dumb agents that worked together to accomplish a goal just like a colony of ants worked together to accomplish a goal The emergent behavior is the sum of individual agent behaviors. There isn t anybody there to decide anything. There s no brain, no higher control in that swarm. [3]

9 Chapter 2 Swarm Simulation 2.1 The Data Structure We implemented a data structure for the swarm and its individuals. The rules were assigned to the individual swarm particles, we called CIndividual. (In this text, we shall use CIndividual and particle interchangeably.) In all the experiments, we made use of a single swarm with a swarm-id and an array of individuals that comprised the swarm. The entire simulation was done using the OpenCV Library of C++ in Ubuntu 9.10 CSwarm { int sname; int* iarray[num] } CIndividual { int mid; int m_x, m_y; float m_vx, m_vy; int r,g,b; // ID of the individual // location on the image (x,y) //its velocity // the r,g,b components of its color } This is the primitive data structure. Depending on the specific goal of the program, some more members (like a Confidence value or the distance_from_the_goal) shall be added to the CInidvidual data structure later.

10 2.2 Initialize a swarm Swarm Intelligence is used in swarm robots to help them achieve their goals. Initially, the robots are kept near each other. Next, they need to self-arrange themselves into random positions such that they satisfy certain minimum and maximum distance constraints. This step is essential for the overall coordination of the swarm-robots, so that, the robots do not go farther than the range of the sensors. This arrangement involves an algorithm called the swarm dispersion algorithm. [4] Here, we simply initialized a swarm with 2 constraints 1. There should be at least one particle in the neighbourhood of another particle. 2. The particles should not overlap, or else the overall number of swarm particles would get reduced. Thus, we initialized a swarm of N particles; we randomly picked up an N (100 in most of the experiments). Correctly initialized swarm Incorrectly initialized swarm Precaution: Make sure that the swarm has a random shape, and not a square one. For that, assign the location of particle at a random location around the particle. In case, we assign the locations of all the particles with respect to the 1 st particle only, the swarm will be of square shape, which looks unrealistic. 2.2 Randomly moving swarm Once we have initialized a swarm, the next step is to start the motion of the swarm, and make it behave like a biological system. We will initialize the various forces amongst the particles, which will result in an emergent random behaviour of the swarm. We shall introduce these forces in the upcoming sections.

11 2.3 Step Response of a swarm To observer the step response of the swarm, we select a peripheral element and give it a velocity and see how the entire swarm responds to this stimulus. On successful simulation of the step response, we observe the subway effect, which is the gradual propagation of information to the swarm particles. To understand the term, imagine a crowded subway. On a crowded subway, when the people at the forefront walk, the entire crowd does not walk simultaneously. It happens gradually. Like initially only first 2 rows start moving, then next 2, then next 3 and so on. It takes some time for the effect of people walking at the forefront to reach the people at the last. This slow propagation of speed is the subway effect. In the same way, under the influence of stimuli, the velocities of the swarm particles changes gradually and hence we observe the subway effect. First step in this process is to find out a peripheral particle and pass on some stimuli to it To find the peripheral particle We shall use the concept of SCAN LINE. To find the leftmost particle of the swarm, we shall start scanning the image. Imagine a vertical line (called the Scan Line) spanning the entire height of the image positioned at x=0. We move this scan line from x = 0 to x = width_of_the_image, until we get a white pixel (or a particle). As soon as the scan line detects this first particle, we break out of the loop, and report the location of that particle to the main code. The red line in the figure indicates the scan line, which travels from left to right and finds out the left peripheral particle of the swarm.

12 2.3.2 The Mathematical Model Once we impart the stimuli to the peripheral element, we need to establish mathematical relationships among the other particles, so that, this stimuli can be propogated. Following is the model followed for this: Let the number of particles in the swarm be N. Region of influence around a particle = (2M+1)*(2M+1) Let be the velocity of the neighbour of a particle, and there are r neighbours around the particle. Then velocity of a particle is given by where Where is the euclidean distance between the particle and its neighbours Important Points: 1. We need to assign some potential (which causes repulsion) at the boundaries. Otherwise, the swarm may go out of the screen leading to Segmentation Fault, OR it may go to the left and emerge from the right of the screen. 2. For keeping track of neighbours, we can introduce a new member in our existing CIndividual data structure, i.e., an array to neighbours, using which we can calculate the neighbourhood adjacency matrix.

13 2.3.4 RESULTS Images 1 6 show the various stages of the Step Response:

14 2.4 Assigning Goal to the swarm The Birds Analogy This system is similar to the swarm of birds searching for food. Each bird is following the primitive rules of attraction and repulsion. In addition, each bird is exploiting the eyes of every other bird, by estimating their distance from the food, and following the one which is closest to the food. The simulation of this efficient approach is the subject of this subsection. Here, we show how we implemented attraction and repulsion among the swarm elements, and their urge to reach the goal Mathematical model Attraction is necessary so to keep the swarm cohesive, otherwise there is a possibility of the swarm getting split into two. Repulsion is needed in order to avoid collision of the swarm particles. And, the urge is required to direct the swarm towards the goal (food). [Attraction factor] [Repulsion factor] [Urge to reach the goal] Here d1 is the distance between the swarm particles, and d2 is the distance of the swarm particle from the goal. The typical values that were suitable for animation were found after a lot of trial-and-error. Thus, the values we used for these constants are And Where and denotes respectively the new and the old location of the swarm-particles. We introduced attraction in the 5x5 neighbourhoods and repulsion in the 3x3 neighbourhoods. Precaution: One problem that may arise during this process is that different characteristics can interfere with each other. Too much attraction can drive to lose track of its goal. Too much repulsion can break the swarm apart. Hence, the values mentioned above came after a number of experiments.

15 2.4.3 Results Our goal coordinates were = (300,200). Interestingly, the swarm takes a V kind of a shape in this simulation.

16

17 2.5 Behaviour in a 2 tank system. In this subsection, we will discuss the modifications needed to help the swarm travel through long narrow corridors. The scenario is as follows: The swarm has to move from region A to region B without colliding with the walls of the region C. For a single agent, we can employ the path finding algorithms like the A* algorithm [10], which is widely used in many computer games. However, for a swarm which learns and evolves, we need a different approach. We define new parameters to carry out this process. We define 2 global variables 1. The centroid of the swarm 2. The radius of the swarm It is the radius of the circle circumscribing the swarm. In the simulation, the first step is detection of the wall. As the wall is coloured grey, its detection is quite easy. Once the wall is detected, the wall potential will not allow the swarm to pass through it. As soon as the swarm reaches the opening of the narrow corridor, it will reduce its radius and travel through the tunnel. Once out of the tunnel, it can go back to its original radius, and reach the target. We can control the radius by varying the values. And, the simulation stops when the centroid of the swarm reaches the target area. For simulation of walls here, we can also make use of the virtual swarm agents [6].

18 Chapter 3 Application in Visual Tracking 3.1 Introduction The concept of swarm intelligence can be used to track objects [5]. We can employ algorithms like predator-prey tracking the objects, wherein the object is the prey and the swarm particles which track it are the predators. The basic idea is that a swarm of particles fly around the image and learns and evolves as the frames go by. The tracking window actually encloses the whole of the swarm, thereby detecting the moving object. We applied the algorithm on the PETS2000 dataset and detected the various moving objects in the dataset. 3.2 Mathematical Model Here we are using the same concept with what we define as CONFIDENCE VALUES. 1. The object to be detected is initialized in the first frame. The entire object is marked white as shown in the figure. These white pixels are actually the swarm elements which will act as tracker pixels. 2. We define confidence value, which represents the confidence level. They were assigned on a scale of 1 to 10. (1 representing min, 10 representing max) Assume that in the frame, we have a red pixel at a position. When the new frame is loaded, we search the neighbourhood of for the same red pixel in the frame. Let denote the number of such red pixels detected in the neighbourhood.

19 While searching, there are 3 outcomes possible: 1. M = 1: which means there is exactly one red pixel in the neighbourhood of the old location of the red pixel. Hence, we have found the new position of the swarm particle representing the red pixel. Moreover, I can calculate the velocity of the swarm particle from this data. And, I set the confidence value of this particle as M = 0: Confidence value is set to zero. The particle is not sure where to go. Hence, in this case it shifts closer to the centroid of the swarm. 3. M > 1: for this case, we use the reduced confidence values. The confidence values are reduced by using the variance of confidence values of the neighbours. After assigning the individual confidence values, we calculate the velocities of individual swarm particle p as, where s are the Confidence values of the particles. The process is repeated for each swarm particle in each new frame, and we observer the emergent object tracking behaviour. (The swarm particles covering the car, which help in object detection.)

20 3.3 RESULTS

21 Chapter 4 Conclusions and Future Work Swarming Pixel Tracker is an intelligent system of tracking objects. However, a lot of improvements and analysis are possible. 1. Stability of swarms is a critical issue. We need to assign proper values of attraction and repulsion factors. A slight incorrectly estimated value may result in the swarm breaking away into fragments, or collapsing into its centre. This issue of stability has although been studies [8, 9], yet it still remains an open problem. 2. We have manually detected the object in the first frame. This detection can be automatized. 3. We found that in various applications, the actual number of particles in the swarm does not matter, as long as the number is large enough to get our goal accomplished. Hence, in our applications we may give rise to or kill the swarm particles. 4. Real-time simultaneity is not easily achievable. This is a limitation of the simulation which can be overcome, if we build better algorithms for swarm simulation. 5. Predicting the exact behaviour of the swarm is also a problem that can be studied. 6. As long as the swarm particles accomplish the assigned goal, the actual number of particles in a swarm does not matter. In many cases, they may die or new ones may take birth. 7. Research is possible in the swarm simulation also. We can study how 2 or 3 swarms interact with each other. That observation will provide us a deeper insight into the swarm behaviour. 8. Swarming algorithms can be used for various life-like simulations. It is here to stay for long. This research can be further expanded towards building a road traffic model.

22 Chapter 5 References 1. Beni, G., Wang, J. Swarm Intelligence in Cellular Robotic Systems, Proceed. NATO Advanced Workshop on Robots and Biological Systems, Tuscany, Italy, June (1989). 2. Reynolds CW (1987) "Flocks, herds and schools: A distributed behavioural model" Computer Graphics, 21(4), Prey by Micheal Chricton 4. MA KOVACINA - SWARM ALGORITHMS: SIMULATION AND GENERATION 5. Swarm Intelligence Based Dynamic Object Tracking by Yuhua Zheng and Yan Meng IEEE Congress on Evolutionary Computation (CEC 2008) 6. Exerting Human Control Over Decentralized Robot Swarms by Zsolt Kira and Mitchell A. Potter 7. Swarm Intelligence: Literature Overview by Yang Liu and Kevin M. Passino, The Ohio State University 8. K. Jin, P. Liang, and G. Beni, Stability of synchronized distributed control of discrete swarm structures," in IEEE International Conference on Robotics and Automation, pp. 1033{1038, 1994} 9. G. Beni and P. Liang, Pattern reconfiguration in swarms {convergence of a distributed asynchronous and bounded iterative algorithm," IEEE Trans. on Robotics and Automation, vol. 12, June