An Improved Evolutionary Algorithm for Fundamental Matrix Estimation

Transcription

1 03 0th IEEE International Conference on Advanced Video and Signal Based Surveillance An Improved Evolutionary Algorithm for Fundamental Matrix Estimation Yi Li, Senem Velipasalar and M. Cenk Gursoy Department of Electrical Engineering and Computer Science Syracuse University, Syracuse, NY 3 yli33@syr.edu, svelipas@syr.edu, mcgursoy@syr.edu Abstract The estimation of the fundamental matrix is an important problem in epipolar geometry. Many estimation methods have been proposed before, including the eight-point algorithm, Simple Evolutionary Agent (SEA) and RANSAC. In this paper, we investigate the evolutionary agent-based algorithm for fundamental matrix estimation, and present a new algorithm that improves the existing evolutionary algorithm both accuracy- and efficiency-wise. The model focuses on selecting a best combination of input points to compute the fundamental matrix via the eight-point algorithm. To improve the existing algorithm, our new model holds competition over all agents for population control and evolutionary experience accumulation. In addition to a larger competition scope, we add the outlier elimination mechanism, which greatly accelerates the algorithm. New parameters are introduced to control the convergence more efficiently. The improved algorithm achieves lower computation load and more accurate results. A general analysis about parameter selection is also provided.. Introduction The estimation of the fundamental matrix is one of the basic problems in epipolar geometry, which is the geometry of stereo vision. Fundamental matrix relates corresponding points in images taken by two different cameras. Given a point in one image, the fundamental matrix narrows down the search for the corresponding point in the other image to a line. Many methods have been proposed in the past for fundamental matrix estimation. A detailed introduction and comparisons are provided for different types of estimation in [] and [8]. Linear methods estimate the fundamental matrix by solving a set of linear equations, and least-squares technique is used to solve over-determined linear equations. Many classical methods belong to this group, such as sixpoint solver [7], seven-point algorithm and eight-point algorithm [5]. The normalized eight-point algorithm [3], which is an improved version of the eight-point algorithm, is a frequently used method. It only requires eight pairs of corresponding points taken from two images. The output of the normalized eight-point algorithm highly depends on the quality of the selected point pairs. In reality, we cannot guarantee that the coordinates of the point pairs are accurate. After feature point detection and pairing, there exists a number of outliers, degrading the performance of the algorithm. In addition, the inlier coordinates are usually affected by noise, which influences the result as well. Robust iterative methods focus on dealing with the outliers. To obtain a reliable result, robust methods aim to eliminate the outliers before applying simple estimators. RANdom SAmple Consensus (RANSAC) [] is one of the most widely used and effective methods to select inliers. Simple Evolutionary Agent (SEA) [] is another robust and iterative algorithm. It uses the idea of random mutation, which randomly changes a small part of the samples to search for a best combination of points that minimize a certain cost function. To avoid being stuck at a sub-optimum solution, several groups search simultaneously. In this paper, we introduce the Evolutionary Agent with Gene Selection (EAGS) method, which is a novel and improved evolutionary agent algorithm. This algorithm forms a gene set consisting of all of the putative point pairs. Then, it uses evolutionary agent to filter the elements in the gene set, and reaches the best group of points, leading to the estimation of a more accurate fundamental matrix. The remainder of this paper is organized as follows. The basics of epipolar geometry, the normalized eight-point algorithm and SEA are described briefly in Section. The details of the proposed algorithm are presented in Section 3. In Section, the impact of the several parameters of the algorithm is investigated via simulation results, and the paper is concluded in Section /3/$ IEEE 6

2 . Background.. Epipolar Geometry Epipolar geometry is illustrated in Fig., wherein a 3D point is projected onto two different image planes. From these two images, we can get the homogenous coordinates of a pair of corresponding points, x = (x,y,) T, andx = (x,y,) T, corresponding to the same 3D point. The line passing through two camera centers is called the baseline. The intersection of the baseline with two image planes gives the two epipoles. The line through the epipole and an image point is the epipolar line. Figure. Epipolar Geometry. Fundamental matrix, F, is a 3 3 matrix with rank, which satisfies the following constraint: x T Fx = 0, () where x and x lie on the corresponding epipolar lines, which are represented byl = Fx, andl = F T x... Eight Point Algorithm Since the coordinates of points contain random, it is impossible to find anf matrix which can satisfy () exactly for all points. Eight point algorithm focuses on minimizing the algebraic x T Fx, which was introduced in [5]. According to the coordinates of eight pairs of points,x i and x i, the algorithm forms an 8 9 matrix as x x x y x y x y y y x y x 8 x 8 x 8 y 8 x 8 y 8 x 8 y 8 y 8 y 8 x 8 y 8. To obtain F, the algorithm just reshapes the generator of the right null space of this matrix to be a 3 3 matrix. Although this F matrix can satisfy () for these eight pairs of points, it does not satisfy the rank constraint. After setting the smallest singular value off to0, the newf matrix will have rank equal to. For the normalized eight point algorithm in [3], we just need to include a normalization process before the eight point algorithm, setting the center of these points to the origin, and the average distance to their center to be. After the algorithm, an inverse normalization is needed to get the F matrix of the original points..3. Sampson Distance To judge the accuracy of a result, we need to compare the average distance or. A lower average distance indicates a better result. Sampson distance [6] is a widely used gradient distance. Consider the fundamental matrix F = f f f 3 f f 5 f 6. f 7 f 8 f 9 The algebraic distance for each point pair isr i = x T i Fx i. Then, Sampson distance is defined as d s = r ( r = r rx +ry +r x +r y ), () where r x = f x + f y + f 7, r y = f x + f 5 y + f 8, r x = f x+f y +f 3,r y = f x+f 5 y +f 6... RANSAC RANSAC is an effective method to select inliers. Since we do not know the exact number of inliers and outliers, the algorithm calculates the fundamental matrix for different numbers of inliers. The final result is the one that minimizes the total function, which is defined as where ρ(d ) = = i ρ(d si), (3) { 0 d < T constant d T, and T is an estimated threshold to distinguish inliers and outliers..5. SEA Evolutionary agent algorithm is another method to select good inliers. Different from RANSAC, SEA finds an optimal combination of eight pairs of points that leads to a fundamental matrix which minimizes the cost function, defined as n D cost = ϕ(d si ), () where and β =.96 ϕ(d) = i= { d d < β β d β, ( ( )) median( di ) (5) n 8 7

3 is a maximum likelihood estimation of the threshold of inliers. The fundamental matrix for cost function calculation comes from the normalized eight point algorithm. Before the evolutionary process, the algorithm builds an index array corresponding to the point pairs. Each agent contains a combination of eight pairs. After initialization, the evolution process starts. In this algorithm, the data structure of agent is as follows: Agent=(index of point pair,...,index of point pair 8, alive indicator, age, cost function). During each evolution loop, each agent is going to produce several child agents. Every child agent inherits the point combination from its parent, and changes several point pairs randomly. If the offspring has better cost function than its parent, it will survive while its parent dies after the reproduction process. If none of the offsprings can do better than their parent, the parent agent will survive, and its age will increase by one. At the end of each evolution loop, the algorithm checks the age of all agents, and very old agents will be weeded out. The evolution loop stops when the agent set is empty, which means there is no alive agent. Finally, the algorithm reaches the best combination that minimizes the cost function. In this algorithm, there are three main parameters that can affect the performance significantly. Parameter mbr is the number of child agents that one parent agent produces in an evolution loop;mch is the number of point pairs that one agent will change; and the initial number of agents. Increasing these parameters can lead to a more accurate result, but it decreases the processing speed. An optimal choice could bembr =,mch =, initial number = Evolutionary Agent with Gene Selection (EAGS) 3.. The Shortcomings of SEA Although simulation results have shown performance improvements of SEA, there are still some shortcomings, which we address to improve this algorithm. First, there is no experience exchange between families, which implies that every family just performs the evolution process independently. If they can share good evolutionary experience, the evolution process would be more efficient. To improve it, we can hold competition among all agents. After competition, we can obtain much more information about the points, and we can further encourage agents to choose points from those good points. Competition is a very good method to exchange evolutionary experience. Secondly, the computational complexity is high. There is no population control in this algorithm, and therefore the population of agents can be very large, which will slow down the processing speed. Moreover, the survival condition is easy to satisfy. To survive, an agent only needs to do better than its parent. Therefore, some useless agents have a chance to survive. 3.. EAGS Algorithm In this section, we present our new algorithm of Evolutionary Agent with Gene Selection (EAGS). The main difference between SEA and EAGS is that EAGS considers the evolution of the gene set instead of the evolution of the individual agent. All point pairs form a gene set, and each pair of points is a gene unit of this set. During the evolution process, every gene unit will gain experience points according to its performance. After several loops, the algorithm will start to weed out some gene units with low experience points. Each agent selects gene units from the gene set randomly instead of inheriting gene units from its parent. This is the evolution of the gene set. In this algorithm, the data structure of the agent is Agent=(index of point pair,..., index of point pair 8, cost function), which is simpler than SEA. After the initialization, the first several evolution loops represent the growing period in which the agent set is in growing mode. During each loop, new agents are produced, which choose gene units from the gene set randomly. Then, the algorithm sorts the agents according to their cost functions. Depending on their rank, only a part of the agents can survive. The gene units in the top 30% of the survivors gain experience points and the gene units of the rest of the survivors gainexperience point while the gene units in the worst30% of the dead agents lose experience points and the gene units of the rest of the dead agents lose point. At the end of each loop, the algorithm updates the best agent. In the growing mode, the population increase is under control and agents select gene units randomly. This period is used to gain enough information about the gene set and identify which units are more likely to be good inliers. After the growing period, the rest of the evolution loops are part of the decline period in which the agent set is in decline mode. During each loop, we first perform the same steps as in the growing mode. Subsequently, we sort all the gene units according to their experience gain. The gene units with small gain in experience points are more likely to be outliers. Every two loops, we weed out 50% of the units with small gain in experience points. This process makes gene set smaller, helping to concentrate on selecting better inliers. After updating the gene set, we reduce the survival ratio in every loop to reduce the population. If the best agent information holds for more than 3 loops during the decline period, the process will come to an end. The algorithm flowchart is shown in Fig.. For this algorithm, there are several important parameters that can have 8

4 significant impact on the performance: Growing factor = Population of(i+)th loop Population ofi th, (6) loop Decline factor = Survival ratio of(i+)th loop Survival ratio ofi th, (7) loop mbr+ = Number of new agents. (8) Total population agent number. mbr works all through the process, decline factor only works in the decline period, and growing factor works only in the growing period. In Section, we investigate the impact of these parameters through simulation results Biology Interpretation In biology, the evolution of a group can be regarded as the evolution of the gene set of the group. The winners of the competition have more advantages to produce offspring, and their gene can be inherited. On the other hand, the losers do not have the right to produce offspring, and their gene may disappear after their death. This process can weed out those bad gene units from the gene set of the group, which represents the evolution of the gene set. In EAGS, the gene units are points. During evolutionary process, bad points are removed and best points will survive at the end.. Simulation Results Fig. 3 shows some examples of image pairs taken from different viewpoints. We obtain sufficient amount of corresponding point pairs with noise and outliers. Here, around 0% of them are outliers. We define the as the cost function of the final result normalized by the number of point pairs, i.e., = Cost function Number of point pairs. (9) Figure. Algorithm flowchart for EAGS. Decline time threshold (DTT) is the time when the system turns into decline mode, and initial number is the initial Figure 3. Examples of image pairs used in the experiments. 9

5 We can regard this as the upper bound of the average Sampson distance for inliers. We evaluate the performance of the algorithms through the average and average. In the following experiments, we vary one parameter and fix the rest to investigate the impact of this parameter. When we fix parameters, we always set mbr as 3, initial number as 7, growing factor as.5, decline factor as0.65 and DTT as8loops. Fig. shows the influence of parameter mbr. When we increase mbr, we can get more accurate results, but at the cost of more. Indeed, we see this fundamental tradeoff between the accuracy and the in the following simulation results as well. Therefore, the optimal choice of the parameter should consider both the accuracy of the result and the Initial number Initial number Figure 5. Initial number versus and initial number versus mbr DTT mbr Figure. mbr versus and mbr versus..5 Fig. 5 shows the influence of the parameter initial number. When we increase the initial number, we get less, but the increases. Fig. 6 shows the impact of DTT. The results show that increasing DTT leads to a more accurate number, but it costs more time. Fig. 7 shows the effect of the growing factor. The results show that increasing growing factor can improve the accuracy, but it also costs more time. Fig. 8 shows the influence of the decline factor. We again see that increasing the decline factor provides more accurate results, but it incurs more time. As the last experiment, we compare the performances of SEA, EAGS and RANSAC. In [], a very detailed comparison between SEA and other improved RANSAC algorithms is provided, and it shows that SEA provides better results with shorter time. Here, we focus on the comparison of SEA, EAGS and RANSAC. To compare them, we use the optimal parameter settings individually for all of them in order to show the best results they can provide. For SEA, DTT Figure 6. DTT versus and DTT versus. we set mbr as, mch as 3, life span as 3 loops, and initial number as 5. For EAGS, we set mbr as 3, initial number as 7, growing factor as.3, DTT as 8 loops, and decline factor as 0.7. To make their results comparable, the result of RANSAC is also based on normalized eight point algorithm. The RANSAC algorithm makes 600 random tests. The average and average are shown in Table. The results show that the improved algorithm EAGS works much better than SEA and RANSAC. The average when using EAGS is smaller than those of SEA and RANSAC, and the average for EAGS is much shorter, providing 70% savings compared to the time that SEA incurs. 30

6 Growing factor Growing factor Figure 7. Growing factor versus and growing factor versus Decline factor Decline factor Figure 8. Decline factor versus and Decline factor versus. avg. (pixels) avg. (s) EAGS SEA RANSAC Table. Average and average of SEA, EAGS and RANSAC. 5. Conclusion In this paper, we have proposed a new algorithm of Evolutionary Agent with Gene Selection (EAGS). Instead of the evolution of agents, this new algorithm is based on the evolution of the gene set, which is significantly more efficient. By holding the competition among all agents, instead of within each family, the algorithm gains more information about the points, and encourages the new agents to focus on good points. This mechanism encourages a large scale of information sharing between each agent and family, which greatly improves the efficiency of searching. Competition also helps to control the population growth, lowering the computational load. With much more efficient evolution and population control, EAGS has been shown to get more accurate results within a shorter time period. Acknowledgment This work has been funded in part by National Science Foundation under CAREER grant CNS-069. References [] X. Armangu and J. Salvi. Overall view regarding fundamental matrix estimation. Image and Vision Computing, :05 0, 003. [] M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, (6):38 395, June 98. [3] R. Hartley. In defence of the 8-point algorithm. In Computer Vision, 995. Proceedings., Fifth International Conference on, pages , 995. [] M. Hu, K. McMenemy, S. Ferguson, G. Dodds, and B. Yuan. Epipolar geometry estimation based on evolutionary agents. Pattern Recognition, ():575 59, 008. [5] H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. volume 93, pages [6] P. D. Sampson. Fitting conic sections to very scattered data: An iterative refinement of the bookstein algorithm. Computer Graphics and Image Processing, 8():97 08, 98. [7] H. Stewnius, D. Nistr, F. Kahl, and F. Schaffalitzky. A minimal solution for relative pose with unknown focal length. Image and Vision Computing, 6(7):87 877, 008. [8] P. H. S. Torr and D. W. Murray. The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, :7 300,