Multisensor Data Fusion Using Two-Stage Analysis on Pairs of Plots Graphs

Transcription

1 Multisensor Data Fusion Using Two-Stage Analysis on Pairs of Plots Graphs Rogério Perroti Barbosa 1,2, Frederic Livernet 3, Beatriz S. L. P. de Lima 1, José Gomes de Carvalho Jr COPPE/ Federal University of Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil 2 Brazilian Navy Research Institute (IPQM), Rio de Janeiro, Brazil 3 General Directorate for Armament (DGA) Toulon, France Abstract This article provides a derivation and a description of the analysis on pair of plots graphs, useful to data fusion of multiple targets in a multiple sensors cluttered environment. The method proposes an analysis in two stages, instead of the previously proposed single-stage method, to choose the best data from possible redundant sensors. The analysis in two stages is parallelizable, which potentially brings performance gains. In this paper, the single-stage and the twostage algorithms are evaluated in light and heavy cluttered environments. The evaluation is based in two different metrics applied over different target trajectories with hard and heavy environmental clutter conditions. Keywords- Target Monitoring; Target fusion; Graph theory. I. INTRODUCTION Command and control naval systems often obtain information from sensors such as radars, infrared sensors, sonars and radio buoys, among others. The signs provided by these sensors may suffer interference from external agents (sea state, weather conditions, etc.), or internal errors (thermal noise, sensor adjustment). These systems must be able to unify the redundant information, providing to the operator a consolidated scenario. This article concentrates in fusion of data provided by different radars. This process is known as multi-target multisensor data fusion. The whole task comprises a tracking estimation method preceded by a plot association process. The conventional Single-Hypothesis Tracking (SHT) [1] algorithm uses the extended Kalman filter (EKF) [2] for filtering and the Global Nearest Neighbor (GNN) [3] for data association. The SHT algorithm propagates only the best data association hypothesis over time. If the association is ambiguous, it fails in providing the hypothesis corresponding to the true association. In a different approach, the track-oriented Multiple Hypothesis Tracking (MHT) algorithm [4, 5] propagates a set of alternative association hypotheses. The least likely hypotheses are discarded, while the true hypothesis is retained. A lot of solutions are similar to MHT and many of these processes are intensive in scenarios with large amount of targets and clutter (false alarms). II. GRAPH-BASED SOLUTIONS The graph based methods are also track-oriented algorithms based on optimizing paths in a graph composed by nodes representing track s positions provided by sensors. The optimizing function varies with model. Some authors [6], [7] proposed fusion methods based on graphs. In these articles each node of the graph represents a target position provided by a sensor in a Cartesian space or a clutter associated with this measure. The data generated by sensors (plots representing target detections or false alarms) are correlated, forming a hypothesis that represents a target path during a certain lag of time. In [8] the authors proposed a different graph-based method, where each node in the graph represents a pair of sequential position points in the Cartesian space. After that, a pair of plots graph is constructed using these points.the authors demonstrated that this approach optimizes the process of filtering clutter associated with sensor measures. To understand the pair of plots graph, consider two sensor providing measurements (sensor 1 and sensor 2). The plots sent by sensor 1 are represented in Fig 1 by circles and plots sent by the sensor 2 are represented by squares. Fig 2 and Fig 3 show graph of plots separated by sensor. They were built using the methodology described in [8], where each plot delivered by the sensor on a time frame is defined as a vertex or node of the graph. Any arc between two plots is built under the following condition: Dpair < Vmax ΔT + f(ε). Where Dpair is the distance between two plots, ΔT is the time elapsed between two plots, Vmax is the maximum speed for the object of interest, and f(ε) is the uncertainty on distance which depends on the measurement errors. In any case this time must obey the following condition ΔT < n*ts where Ts is the sensor update rate and n is the number of time steps in the current window. From the graphs of Fig 2 and Fig 3, are elaborated the pairs of plots graphs shown in Fig 4 and Fig 5, respectively. Those graphs are constructed using the speeds and times of points that belong to each arc in the plot graph to be a new node in the pairs of plots graph. The previous nodes are discarded. So, a new graph is built where the nodes of the final graph are the arcs of the previous graph. 2073

2 Figure 1 Plots (sensor 1 and 2). Figure 2 Graph of plots (sensor 1). In the pairs of plots graph, the arcs represent the several hypotheses for targets paths on a selected time window. After the graph construction, the hypotheses are evaluated and an optimization process is fired to choose the most probable path for target based in the Dantzig s algorithm [9]. Dantzig s algorithm was developed to determine and rank the different paths of the graph according to a predefined criterion. For instance, in the famous travelling salesman problem, the criterion is the distance between the different cities. According to [8], one key point of the pair of plots graph based tracking is the estimation of kinematics consistency criteria between the arcs of the graph. These criteria are used by the recursive optimal path cover algorithm such as Dantzig s algorithm to determine the most consistent hypothesis /path among the possible one. The kinematics criteria used by the Dantzig s algorithm is based on the following basic data: The speed of each node of the graph: Where t and x are the time and position of the plots of the node i. The acceleration between two vertices of each arc is: Figure 3 Graph of plots (sensor 2). Figure 4 Pairs of plots graph (sensor 1). Where V xi and V xi-1 are the speed of the two vertices. Using the two kinematics data described above, the criteria used by the Dantzig s algorithm defined in [8] is presented following in equation (1): 1 (1) Where N is the number of arcs of any hypothesis or path, τ is the variation of acceleration between two consecutive arcs of a path, and Gi is a normalization factor. Dantzig s algorithm determines the hypothesis /paths of the graph that minimize C i.e. the one that contain a maximum number of minimize C i.e. the one that contain a maximum number of vertices while minimizing mean square acceleration deviation. The methodology proposed in [8] applies the Danzig algorithm over a pair of points graph constructed with all points supplied by all sensors involved in the fusion process. As the number of sensors increases or as the clutter becomes heavier, the number of hypothesis grows rapidly, impacting the performance. Figure 5 Pairs of plots graph (sensor 2). III. THE PROPOSED SOLUTION This article proposes another approach to construct the pair of plots graphs that enables paralyzing the optimization 2074

3 process. Instead of constructing a single graph for all the sensors, the method proposes two steps. In the first step, the method constructs a pair of plots graph for each sensor and applies the optimization process for each graph (Fig 6). This approach enables the parallelism of this first step. After constructing the new pair o plots graph with only the plots that were not pruned in the first step, the optimization process using Dantzig algorithm is repeated. The final result is the best global hypothesis for target path, considering all the sensors (in this case, only two sensors) plots and clutter involved in the acquisition process (Fig 9). Figure 6 Trajectories chosen by the first step. In a second step, another graph is constructed, now using the optimized paths selected from each sensor pairs of plots graph from the first step (Fig 7). Figure 7 Optimized plots graph. In this new graph the nodes also represent plots. However, only the nodes selected as forming the best hypothesis for each sensor in the first step are considered to construct the new hypothetical paths. The nodes are not necessarily the same used in the first step. The new graph is constructed with new nodes that represent potentially different associations between plots (Fig 8). Since the sensors are not considered to be synchronized, the time interval between plots may vary. Thus, a pair of plots in this stage can associate plots from the same sensor or from two different sensors, depending on the time they are obtained. However, the plots considered to form a new pair are those that originally form a pair that was not pruned in the first step. Figure 8 - Pairs of plots graph (sensors 1 e 2). Figure 9 Best trajectory, sensors 1 and 2. The reduction in the number of hypothesis in the first step is accompanied by the potential parallelism of this step, bringing significant savings in time, as related in section IV. In addition, the second step constructs a graph with a significant decrease in the number of nodes, which also accelerates the optimization process. These time savings, however, are not obtained against a loss of accuracy in the whole process. On the contrary, there were gains in terms of accuracy, as shown by the metrics proposed following. IV. TESTING SCENARIO A simulation software was created to validate the proposed model. This software provides a model to simulate targets in a traffic transit area accordingly to a specific user-defined cinematic. It is also possible to define sensors to provide data acquisition data from traffic area scenario. The software also simulates data sensor acquisition process, including targets detection plots and false plots due to clutter. All data generated by data sensor acquisition process is used as input for the fusion algorithm to operate on. The targets trajectories in the traffic area are simulated considering a user-defined cinematic and a Gaussian process noise. Each data sensor acquisition process adds one Gaussian noise for that particular sensor. This noise is added to the spherical coordinate s measurement accordingly to a desired sensor noise variance in range and bearing measurement. The standard deviation values for each input noise are configurable in the simulation software. The time interval between each sensor update is also configurable, besides a probability reflecting the tax of plot losses for each sensor in the data acquisition process. After plots provided by data acquisition process for each sensor are available, other plots are added to the data input set, representing clutters in the acquisition process. These false plots are generated with a uniform distribution around the track plot position with the maximum number of clutters and distance to the real plot defined by the user. 2075

4 The simulated data used in tests are presented in figures like Fig 10. In this figure plots sent by sensor 1 corresponding to actual vessel detection are represented by x. The same plots provided by the other sensor are represented by +. Plots representing clutter for both sensors are presented as. in the graph. It may also be observed, in Fig 10, the presence of the correct position of the vessel, calculated by the simulator before the introductions of process noise and acquisition data noise, represented by diamonds in the graph. Figure 10 Simulated Data. The fusion algorithms receive all these plots, obviously without distinction, to perform filtering and fusion. The algorithm output is the plot representing the best hypothesis for vessel position at that specific time. This final plot is represented as a circle in the graph (Fig 11). So, circles with x or + inside them represent right choices, while circles with. inside represent wrong choices (clutter choices). path of the target. This metric shows a ratio of distances that can show a deviation of the chosen route in relation to the most probable route of the target. The most probable route contains in each instant of time points of minor deviation from the real trajectory. This metric is called Distances ratio (M2): 2 min, Where is the distances between points chosen and points that best represent the path of the target, and n is the number of time windows during test trajectory. Two different target traffic scenarios and two different data acquisition conditions were created to test the model. In the first scenario, a single target is modeled with a speed of 1000 km/h. After performing a fifty seconds straight run, it turns starboard with a turn rate of 4 degrees/sec during 45 seconds and finally straight run for another fifty seconds (Fig 12). In the second scenario, a single target is modeled with a speed of 500 km/h. After performing a fifty seconds straight run, it turns starboard with a turn rate of 4 degrees/sec during sixty seven seconds and finally straight run for another fifty seconds (Fig 13). Data provided by two sensors are used for all the tests. In the first data acquisition, a light clutter environment is simulated, with a random amount of false alarms per sensor acquisition time varying from zero to 10 plots. In the second data acquisition, a heavier clutter condition is simulated, with amount of plots varying till 30 false alarms per sensor acquisition time. Both scenarios are simulated with light and heavy data acquisition conditions. Figure 11 -Plots after filtering and fusion Two metrics were proposed for algorithms evaluation: 1. The mean percentage of points chosen correctly inside the time window (M1). 1 / 100 Where n is the number of time windows during test trajectory, A is the number of true choices in a time window, and T is the number of chosen points in time window. 2. Relationship between the distances of points chosen and the distances of the points that best represent the Figure 12 Trajectory 1 (Heavy Clutter) Figure 13 Trajectory 2 (Heavy Clutter) 2076

5 V. RESULTS OBTAINED USING THE TWO METHODS Since comparison between the pair of plots (PP) graph method and the classic Multi Hypothesis Tracking (MHT) was done in [8], resulting in gains for the first method over the second, this work only compares the proposed method, called pairs of plots in two stages (PPTS) with the PP method. Tables I, II, III and IV show the results obtained applying the PP and PPTS methodologies. In the trajectory 1 with light clutter (TABLE I), the percentage of correct points (M1) is the same using the two methods, but the second metric (M2) demonstrates that the points chosen by PPTS are closer to the real points, meaning that PPTS made better choices. The computational time (T) is lower with PPTS, allowing the selection of a greater time window, which is essential in complicated scenarios. TABLE I. RESULTS IN THE FIRST TRAJECTORY USING LIGHT CLUTTER Trajectory 1 with light clutter Segment 1 M1 100% 100% M T 0.79s 0.33s Segment 2 M1 100% 100% M T 0.86s 0.34s Segment 3 M1 100% 100% M T 0.75s 0.37s In the trajectory 1 with heavy clutter (TABLE II), the percentage of correct points was slightly greater on the first segment with PPTS, and in the other two segments PPTS obtained the same tax of correct points as PP (100%). Similar to the first case, the second metric demonstrates that the points chosen by PPTS are closer to the real points, meaning better choices. The computational time is much lower using PPTS in all three segments of the trajectory, showing the possibility of using greater time windows, which can improve accuracy. TABLE II. RESULTS IN THE FIRST TRAJECTORY USING HEAVY CLUTTER Trajectory 1 with heavy clutter Segment 1 M % 99.40% M T 13.16s 2.54s Segment 2 M1 100% 100% M T 14.58s 3.05s Segment 3 M1 100% 100% M T 12.36s 4.13s In the trajectory 2 with light clutter (TABLE III), the percentage of correct points was greater in the first and the third trajectory s segments using PP, and higher in the second segment using PPTS. Like the previous test cases, the second metric demonstrates that the points chosen by PPTS are close to the real points, meaning again better choices. The computational time was smaller with PPTS like in trajectory 1 making possible use of greater time windows. TABLE III. RESULTS IN THE SECOND TRAJECTORY USING LIGHT CLUTTER Trajectory 2 with light clutter Segment 1 M % 83.86% M T 1.09s 0.45s Segment 2 M % 97.81% M T 1.01s 0.45s Segment 3 M % 87.61% M T 0.87s 0.39s In the trajectory 2 with heavy clutter (TABLE IV), the percentage of correct points was greater in the three trajectory segments, and once more the second metric indicates that PPTS made better choices. The computational time was reduced more than six times, making possible the use of a big size time window in real time. TABLE IV. RESULTS IN THE SECOND TRAJECTORY USING HEAVY CLUTTER Trajectory 2 with heavy clutter Segment 1 M % 90.90% M T 25.9s 2.68s Segment 2 M % 89.94% M T 31.29s 5.20s Segment 3 M % 83.33% M T 84.55% 90.90% VI. CONCLUSIONS According to [5], the pair of plots (PP) graph analysis represents an important alternative in track to plot association process, mainly in comparison with traditional MHT algorithm. In this article, an improved version of PP algorithm, named pairs of plots in two stages (PPTS) was presented, in order to decrease the number of association hypothesis investigated and increase the performance by paralyzing some algorithm steps. In order to evaluate the results, two metrics were proposed. The first one is the percentage of correct points chosen by the algorithm in a cluttered environment. The second metric is the deviation of the chosen route from the most probable route of the target. Tests realized with simulated scenarios demonstrated that PPTS made better choices among points provided by simulated sensors, resulting in more accurate estimated trajectories. The 2077

6 proposed algorithm is especially efficient, as demonstrated by the results, in environments with high density clutter, allowing more precise solutions in the target tracking and effecting the coherent fusing of data proceeding from many sensors, as shown in Tables I, II III and IV. Besides performing better in both proposed metrics, PPTS also provide reduction in computational time. It can be noted that gains obtained in time performance of PPTS over PP were measured without using the paralyzing possibilities provided by PPTS, since multi thread resources were not available in the testing implementation environment. The use of this resource will probably increase the difference in computational time performance. Time gains obtained with the proposed algorithm are proportional to the number of sensors considered in the fusion process, since data provided by each sensor may be treated in a parallel thread in first stage of PPTS algorithm. Thus, in multi sensor environments, the computational time difference between PP and PPTS tends to increase if a multi-thread computational environment is available for PPTS execution. Time gains obtained with PPTS may also be used to increase the size of past time windows, enabling to work with more past points and providing better accuracy in pair of plots association algorithms, according to [5]. It should also be highlighted that the significant reduction of time enables application of PPTS in real time target tracking environments, monitored for many sensors in complex scenarios with great number of targets, multi sensors and high noise. REFERENCES [1] A. Jazwinski, Stochastic Processes and Filtering Theory. New York: Academic, 1970, ch. 8, pp [2] Y. Bar-Shalom, and Xiao-Rong Li, Multitarget-Multisensor Tracking: Priciples and Techiniques, [3] S. Blackman and R. Popoli, Design and Analysis of Modern Tracking. Systems. Norwood, MA: Artech House, [4] D. Reid, An algorithm for tracking multiple targets, IEEE Transactions on Automatic Control, vol. 24, no. 6, pp , December [5] S. Blackman, Multiple hypothesis tracking for multiple target tracking, IEEE Aerospace and Electronic Systems Magazine, vol. 19, no. 1, pp. 5 18, January [6] Y. C. Tang e C. S. G. Lee, A geometric feature relation graph formulation for consistent sensor fusion, in Systems, Man and Cybernetics, Conference Proceedings., IEEE International Conference on, 1990, p [7] D. L. Hall and J. Llinas, Handbook of Multisensor Data Fusion: Theory and Practice, CRC Press, [8] Frédéric Livernt, Olivier Cuillier, Tracking based on graph theory applied to pairs of plots, Radar Conference, IEEE. [9] Yves Tabourier, All shortest distances in a graph. An improvement to Dantzig s inductive algorithm, Discrete Mathematics 4,North Holland Publishing Company,