Radar Detection Improvement by Integration of Multi- Object Tracking

Transcription

1 Radar Detection Improvement by Integration of Multi- Object Tracing Lingmin Meng Research and Technology Center Robert Bosch Corp. Pittsburgh, PA, U.S.A. Wolfgang Grimm Research and Technology Center Robert Bosch Corp. Pittsburgh, PA, U.S.A. Jeffrey Donne Research and Technology Center Robert Bosch Corp. Pittsburgh, PA, U.S.A. Abstract - This paper presents a new and simple approach to the problem of multiple sensor data fusion. We introduce an efficient algorithm that can fuse multiple sensor measurements to trac an arbitrary number of objects in a cluttered environment. The algorithm combines conventional Kalman filtering techniques with probabilistic data association methods. A Gauss Marov process model is assumed to handle sensor outputs at various sampling frequencies and random nonequidistant time intervals. We applied the algorithm to post-process the digital range returns of radar sensors to improve their quality. Since the static noise returns have near-zero velocity, the algorithm associates a certain trac with each digital return, and estimates the trac velocity, thereby allowing for removal of false returns originating from static pattern noise. Keywords: multiple object tracing, sensor data fusion, estimation, data association, radar, detection, static noise. 1 Introduction Ordinary detection systems only consider the information contained in a single frame or scan. Usually, the objects to be detected have an intrinsic model or property across multiple scans or frames. Combining the object model with the information of each scan or frame can certainly improve the detection result. It is worth mentioning that there exist some algorithms integrating detection with target tracing, such as PDAF-BD [1], in which the detection threshold varies according to the ratio of prior probabilities. However, it only handles one trac. We developed a generic multiple sensor multiple object tracing algorithm and applied it to improve radar detections. It exploits the nature that radar detections of moving objects are consistent over time while spurious noise is fluctuating and unstable. In particular, we successfully removed the static noise by estimating and recognizing its near-zero velocity. Many approaches of tracers have been investigated and widely used. The main difference lies in the data-trac association method. The probability data association (PDA) [2] is a particularly simple and successful target tracing algorithm. It maintains only one trac, computes observation-to-trac assignment probabilities and then updates the trac with a probabilistically weighted composite of all observations within the trac gates. Later, a modified method, the joint probabilistic data association (JPDA) [3], was derived to include the presence of multiple targets. Multiple hypothesis testing (MHT) [4] defers the decision, and, rather than combining these hypotheses, the hypotheses are propagated in anticipation that subsequent data will resolve the uncertainty. Though similar to PDA and JPDA, our tracing algorithm has the following features: It can maintain a variable number of tracs. Each observation data is independently evaluated and associated to only one existing trac or to form a new trac. Based on the maximum a priori (MAP) rule, each data is selected to be used solely to update a trac. 2 Tracing algorithm with probabilistic data association We derive the algorithm from the simplest case: a single sensor tracing a single object, in a statistical formulation. 2.1 Single object tracing by a single sensor Let us assume the standard discrete-time description of the process model and observation formulation : x( t ) F( t 1 t ) x( t ) u( t 1 (1) z t ) Hx( t ) v (2) ( ) 1249

2 where t ) x is the object state to be estimated and z t ) ( is the measurement at the -th sampling time. Both the transition matrix F and observation matrix H are assumed to be nown, though F depends on the time interval between -th sample and (+1)-th sample. The process noise u and measurement noise v are assumed to be independent zero-mean Gaussian-distributed with variance matrices Q and R respectively, where Q usually depends on the time interval between the -th sample and the (+1)- th sample and R is independent of time. We assume that both Q and R are nown. An optimal estimator would be the Kalman filter which taes the observation (or sensor measurement) as input and dynamically updates the state estimation and confidence estimation. For Gaussian variables, the state estimation is the mean and the confidence estimation is the covariance matrix. 2.2 Problem formulation Tracing becomes more challenging, when the system has multiple sensors measuring multiple objects at various sampling frequencies and non-uniform time intervals. The output of each sensor can correspond to any or no objects. The time each sensor measurement occurs can be random and independent of other sensors. We assign a Gauss-Marov process for the i-th object being traced: i i x ( t 1) F( t 1 t ) x ( t ) u( t ) (3) z ( s s s ( t ) H x( t ) v (4) where i is the object number and s is the sensor number. However, each observation z(t) from any of the sensors is not explicitly associated with an object, hence, we do not now to which of the Kalman filters to feed the observation. Even worse, the observation may be from noise and should not be associated to any object. 2.3 Multi-object tracing and data association We provide a generalization of the Kalman filtering to multiple target tracing and multiple sensor fusion. The ey to the solution is (i) using tracing information to perform data association and (ii) using data association to update tracing. Specifically, at each time instant as a batch of new observations become available, we match each new observation to all the traced objects via statistical models and then only update the best-matched object. We also separate the time update from the observation update. As a result, the sensor synchronization is no longer a problem and there is no need for sequential prefiltering for each individual sensor. To ensure that data are observed independently, all data observed at the same time should be evaluated at the same time, then the tracs are updated afterwards. Let denote the traced object number to which the current observation z belongs. For simplicity, we do not have an explicit clutter model. Instead, we assign a new object if the observation can not match any of the existing objects. Then it will be considered as noise by subsequent observations. Let = denote the case that the observation is a new object. There are many methods commonly used for data association. We use the simple Maximum A Posterior (MAP) rule to choose the associated object number from the pool of traced objects. * ( z) arg max p( z) (5) {,1,2...} Then, we use z to update the estimates of object * ( z ). The posterior probability can be computed by using Bayes Rule, as (6) f ( z, ) f ( z ) p( ) p( z) f ( z) f ( z p( where p(=) is the prior probability of being object without the observation z, and f(z =) is the lielihood probability density value that the observation z belongs to object. For simplicity, we assume p( ) is a constant. For any data z, the MAP rule is essentially the maximum lielihood selection: * ( z) arg max p( z ) (7) {,1,2...} Hx v ), P ( t t )) Recall that z where v N (, R ) and x N ( xˆ ( t t. Further, x ˆ( t t ), P ( t t ) are the estimated mean and variance of object at time t using information prior to time t without the observation z. Therefore, z N ( H xˆ ( t t ), HP ( t t ) H T R ) (8) We usually assign a very small number to f(z =) if we assume uniform distribution for new objects. Sometimes this value is difficult to compute. In effect, parameter determines how often an observation is assigned to be a new object. Thus, one can tune the parameter by trial and error until the new trac generation rate reaches the desired level. During the Kalman filtering process, the variance estimation always increases in the prediction step and 125

3 decreases in the observation update step. The maximum achievable value ( maximum when the data is the expected mean z Hx ( t t ) ) of p( z ) depends on the determinant of the covariance matrix HP ( t t ) H T R. As P( t 2 t ) P( t1 t ) if t2 t1, the determinant of T HP ( t t ) H always increases without new observation updates. Thus, once the maximum achievable value of p( z ) drops below parameter, no data can ever be matched to the trac again. This trac will be terminated to reduce unnecessary further computations for unliely observation-to-trac assignments. identity matrix and R( is diagonal if the errors in (x,y)- positions and velocities are independent. The assumption of this process model is that the velocity of each object is constant during the time interval T. This inaccuracy is considered additional process noise u( which is assumed zero-mean Gaussian with nown diagonal matrices. x( n 1) 1 y( n 1) vx( n 1) vy( n 1) 1 T n1 1 x( u1( T n1 y( u2( vx( u3( 1 vy( u4( (9) If two observations occur at the two time instants t1, t2 (t1<t2), we assume that the observation data z(t1) is always received by the real-time processor before z(t2). Therefore z(t1) is fed to the algorithm before z(t2). The algorithm is summarized by the following steps : 1) Initially, the trac pool is empty. 2) Collect all the observations at time instant t. 3) Perform a prediction step for each trac in the pool, drop those tracs which have a maximum achievable lielihood below the threshold. 4) For each data, evaluate the lielihood probability for each of the tracs currently in the trac pool. 5) For each data, update the trac which fits best (highest lielihood) or form a new trac if all trac lielihoods are below the threshold. It is possible that a trac is updated at the same time instant by more than one observation data. 6) Go to step 2) for next time instant. Multiple object tracing is well suited for object-oriented programming concepts. We call each traced object a trac. A trac instance is generated whenever a new measurement is assigned to be a new trac. The first measurement is always assigned to be a new trac. A trac can be updated by time or by an assigned measurement, requiring two methods: timeupdate and observationupdate. A trac is deleted once the maximum achievable lielihood drops below. 3 Experiments 3.1 Simulation of multiple sensor fusion A common effort is to use several low-performance sensors and apply sensor fusion algorithms to achieve a comparable quality of a single expensive sensor of higher performance. We first tested the algorithm by simulation, where three independent sensors are monitoring three moving targets. The process model of equation (9) and the observation model of equation (1) are used. The observation vector z( denotes (x,y)-positions and (x,y)- velocities. Therefore, H( in equation (2) becomes an 1251 z ( n ) Ix ( n ) v ( n ) (1) A simulation result is shown in Fig. 1, where three tracs are generated using the same process model. Random observation noise is added to the output of each sensor. One of the sensors has only half the sampling rate of the other two sensors. Figure 1 compares the simulation results of using different data sources. Clearly, we achieved a more accurate result (bottom right) when using all available data than by using measurements from only one sensor Figure 1. Each graph shows the (x,y)-locations. The first column from top to down shows observations from sensor 1, 2, 3, and the true object tracs respectively. The second column from top to down shows fusion results using only data from sensor 1, 2, 3, and from all three sensors respectively. Each color represents a trac.

4 3.2 Radar detection improvement Although the algorithm is proposed to perform sensor data fusion, it can also be applied to process single sensor measurements because one still needs to solve the observation-trac association problem as long as multiple objects are present. We were experimenting with several mono-static pulsed radar units. Due to the limited antenna size of some of the radar units, the limited signal processing capabilities using relatively cheap microcontrollers, as well as some other reasons, the detection of some of the units turned out unreliable. Figure 2 shows a typical interim analog output of the matched filter [5] of some of the sensors under investigation. The radar signal processing unit processes the output of the matched filter and always returns a definite number of discrete range outputs each with a time stamp regardless of the number of targets that are present within the detection field. No confidence measurements or amplitude information were used. A range output value of zero indicates no detections. These outputs are usually heavily contaminated by some false returns around some fixed values, which we call fixed pattern noise. It seems that the undesired performance of some of the radar units is neither due to multiple reflections from walls or the ground nor due to interferences with other radar sensors. Figure 3. Fixed noise return of first radar unit: range vs. time. Note the value at 2.4, as well as clusters around 4.4 and 6.3. The x-axis is the time index, the y-axis is the range measurement. Figure 4. Fixed noise return of second radar unit. Note the value at 2.4, as well as clusters around 5.9, 6.9, and 7.2. Figure 2: Typical echograms with only one object present. In-phase (left) and quadrature channel (right). The performance of two different radar sensors is shown in Figure 3 and 4 respectively, where the detection output is plotted versus time. Although no objects are present in the vicinity of the sensors, one may notice fixed noise, which is more severe for the second sensor (Figure 4). The outputs of the radar sensors under investigation may drift due to nearby objects even for the same radar unit. The unreliable performance is illustrated in Figure 5, where an object moves towards the sensor used for Figure 4. Figure 5. Return of second radar unit for the case of an object moving towards the unit. Note the location shifts (around range measurement 6.) of the fixed noise due to the presence of objects compared with Figure 4. A simplistic approach was initially tried, which wipes out the returned range values in certain intervals which can be obtained by training a Gaussian Mixture Model (GMM) from the fixed pattern noise data. Figure 6 shows one example. However, there are immediately two disadvantages for this method: (i) it requires off-line training or calibration to determine the locations of fixed noise clusters because different sensors have different 1252

5 characteristics; (ii) the true objects within the noise interval will also be wiped out. Figure 6. Top to bottom: (a) raw data from radar sensor, (b) output after removing the fixed value at 2.4, (c) output after removing clusters near 4.5 and 7. To tacle these two problems, we utilize the previously developed multiple-object tracing algorithm to process the digital output of each radar sensor to remove the static noise. This algorithm may not be optimal compared with JPDA or MHT, but it is simpler and more appropriate to implement in real-time under computational constraints. The basic idea is to distinguish between static noise and moving objects (relative to the sensor or the vehicle) by labelling data with tracs and estimate the velocity of each trac. The basic assumptions are: Spurious noise measurements are sparse and are not consistent through time; the tracs formed by spurious noise have lower association probability because the estimated variance is large due to the lac of enough observation updates. Static noise measurements persist in the tracing volume, but the tracs formed by static noise have near zero velocity. The measurements of moving objects (along range directio persist in the tracing volume for at least several scans, with non-zero velocity. The input to the algorithm is a sequence of detection ranges with time stamps, noted as x i (t), where t= t 1, t 2, t 3, t 4,..., i=1,2,3. To expedite the process, the fixed constant values (e.g. 2.4 in Figure 3) are easily recognized and removed by an algorithm. The state variables are the location x and velocity v x. We choose the state transition matrix and the process noise variance matrix respectively as: t F ( t) ; 2 t / 4 t / 2 Q ( t ) q t / 2 t where t is the time increase and q 2 is a tunable parameter representing the random behavior of the moving object [2]. The algorithm outputs the estimates of range, velocity, and their variances and the association probabilities for each input sample: xˆ ( t), vˆ ( t), 2 2 vˆ ( t), x ( t), v ( t), f ( z ), t= t 1, t 2, t 3, t 4,..., =1,2,3,...,N(t), where N(t) is the total number of objects being traced at time t. The variance is inversely proportional to the confidence of the estimate. From the estimated ranges, we choose those that satisfy a joint condition as true range detections. The joint condition is: vˆ ( t) T1 and f ( z ) T2. That is, the velocity is above threshold T 1 and its association probability is above threshold T 2. Thresholding the velocity variance estimation is an alternative to thresholding the association probability. Based on our experiments, the latter achieved better detection vs. false alarm performance. An explanation could be that the variance estimation depends only on the prediction time and the number of data updates but does not reflect how well the data is fitted to the trac. The choice of the threshold and the estimate of process 2 noise q has a significant effect on the tracing performance. We determined the parameters by trial and error until the best result was achieved. Fortunately, these parameters have to be determined only once for one series of data, and they can be applied to other data to achieve comparable results. In Figure 7, 8, and 9, the detection process and its final result is shown for two different radar units. Clearly, most of the fixed pattern noise is removed (see Figure 7f and 8f). Note that even though the moving object moves into the static noise interval, it is still detected due to its nonzero velocity (Figure 7f). Note that a failure in detecting an object may occur when its velocity is very close to zero (Figure 8f). Further algorithms have to be applied to deal with this situation. 4 Conclusions In this paper, a simple multi-sensor multi-object tracing algorithm is developed. The algorithm combines Kalman filtering with probability data association methods. Using this algorithm, we successfully removed static fixed pattern noise from radar outputs. Since we used the algorithm mainly for detection purposes, we found the single most liely hypothesis for the assignment of observations to existing tracs or to new 1253

6 trac initiation. The other choice, a little more complex, is to associate each data to several of the current tracs, and update each trac using a weighted combination of all data according to their association probabilities. Even more complex, we could apply JPDA or MHT to investigate if the performance can be further improved. pattern noise. To further improve the algorithm to include the non-moving object detection, we suggest to integrate statistics of the history of each moving and static trac, so that the system can intelligently tell if the missed part is a real static noise trac or belongs to a moving object coming to a stop. Figure 8. Example of radar range filtering of unit 2. Figure 7. Example of radar range filtering of unit 1: (a) raw detection output from radar unit. (b) input data for the tracing algorithm, after continuous fixed values are removed. (c) tracing and data association result, where each color corresponds to a trac. (d) the data association probability value, where each color corresponds to a trac. (e) estimated velocity of all tracs at each observation point, where each color corresponds to a trac. (f) range detections of moving objects are mared red, while bacground fixed noise (mared in gree will be filtered out. As shown in Fig. 8, the object is mis-classified as static noise by the algorithm when it is not moving. In theory, the algorithm with a first-order inematics model is not capable to distinguish a stationary object from static Figure 9. Blow-up portion of Figure

7 The authors would lie to than Mel Siegel and Robert MacLachlan at the Robotics Institute of Carnegie Mellon University for data collection and helpful discussions. References [1] P. Willett, R. Niu, and Y. Bar-Shalom, Integration of Bayes Detection with Target Tracing, IEEE Transaction on Signal Processing, vol. 49, no. 1, pp , Jan. 21. [2] Y. Bar-Shalom, X.R. Li, Estimation and tracing: Principles, Techniques and software. Artech House, Boston, [3] T.E. Fortmann et al., Sonar Tracing of Multiple Targets Using Joint Probabilistic Data Association, IEEE J. of Oceanic Engineering, Vol. 8, July 1983, pp [4] S. Blacman, R. Popoli, Design and Analysis of Modern Tracing Systems, Chapter 6, Artech House, Boston, [5] P.Z. Peebles, Radar Principles, John Wiley&Sons,