Lecture Outline. Target Tracking: Lecture 6 Multiple Sensor Issues and Extended Target Tracking. Multi Sensor Architectures

Transcription

1 REGLERTEKNIK Lecture Outline AUTOMATIC CONTROL Target Tracing: Lecture 6 Multiple Sensor Issues and Extended Target Tracing Emre Özan emre@isy.liu.se Multiple Sensor Tracing Architectures Multiple Sensor Tracing Problems Extended Target Tracing Division of Automatic Control Department of Electrical Engineering Linöping University Linöping, Sweden December 7, 4 E. Özan Target Tracing December 7, 4 / 53 E. Özan Target Tracing December 7, 4 / 53 Multi Sensor Architectures Multi Sensor Architectures Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. E. Özan Target Tracing December 7, 4 3 / 53 E. Özan Target Tracing December 7, 4 4 / 53

2 Multi Sensor Architectures: Centralized Multi Sensor Architectures: Hierarchical Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. E. Özan Target Tracing December 7, 4 5 / 53 E. Özan Target Tracing December 7, 4 6 / 53 Multi Sensor Architectures: Hierarchical with Feedbac Multi Sensor Architectures: Decentralized Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. Figures taen from: M.E. Liggins and Kuo-Chu Chang Distributed Fusion Architectures, Algorithms, and Performance within a Networ-Centric Architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. E. Özan Target Tracing December 7, 4 7 / 53 E. Özan Target Tracing December 7, 4 8 / 53

3 Multi Sensor Architectures: Pros & Cons Problems in Multi Sensor TT The traditional centralized architecture gives optimal performance but Requires high bandwidth communications. Requires powerful processing resources at the fusion center. There is a single point of failure and hence reliability is low. For distributed architectures Communications can be reduced significantly by communicating tracs less often. Computational resources can be distributed to different nodes Higher survivability. It is a necessity for legacy systems e.g. purchased radars might not supply raw data. Registration: Coordinates (both time and space) of different sensors or fusion agents must be aligned. Bias: Even if the coordinate axis are aligned, due to the transformations, biases can result. These have to be compensated. Correlation: Even if the sensors are independently collecting data, processed information to be fused can be correlated. Rumor propagation: The same information can travel in loops in the fusion networ to produce fae information maing the overall system overconfident. This is actually a special case of correlation. Out of sequence measurements: Due to delayed communications between local agents, sometimes measurements belonging to a target whose more recent measurement has already been processed, might arrive to a fusion center. E. Özan Target Tracing December 7, 4 9 / 53 E. Özan Target Tracing December 7, 4 / 53 Correlation Correlation Centralized Case y Fusion Center y Decentralized Case y Tracer- ˆx ˆx Fusion Center ˆx Suppose the fusion center have the prediction ˆx in both cases. Centralized Case [ ] y = y [ C C ] x + [ e e where e and e are independent. Decentralized Case [ ] ˆx [ I = I ˆx ] x [ x x ] ] Suppose the target follows the dynamics x = Ax + w and the ith sensor measurement is given as y i = C ix + e i Then with the KF equations ˆx i =Aˆxi + Ki (yi C iaˆx i ) =Aˆx i + Ki C i(x Aˆx i ) + Ki ei =Aˆx i + Ki C ia(x ˆx i ) + Ki C iw + K i ei y Tracer- ˆx E. Özan Target Tracing December 7, 4 / 53 E. Özan Target Tracing December 7, 4 / 53

4 Correlation Correlation Define x i x ˆx i, then x i =x Aˆx i Ki C ia(x ˆx i ) Ki C iw K i ei Hence =Ax + w Aˆx i Ki C ia(x ˆx i ) K i C iw K i ei =(I K i C i)a x i + (I Ki C i)w K i ei x i =(I Ki C i)a x i + (I Ki C i)w K i ei x j =(I Kj C j)a x j + (I Kj C j)w K j ej We can calculate the correlation matrix Σ ij E( xi xjt ) as Σ ij = (I Ki C i)aσ ij AT (I K j C j) T + (I K i C i)q(i K j C j) T Assuming that Σ ij =, we can calculate the correlation between the estimation errors of the local tracers recursively as Σ ij = (I Ki C i)aσ ij AT (I K j C j) T + (I K i C i)q(i K j C j) T This necessitates that the fusion center nows the individual Kalman gains K i and Kj of the local tracers which is not very practical. Assuming that the errors are independent is not a good idea either. Neglecting the correlation maes the resulting estimates overconfident i.e., very small covariances meaning that too small gates and smaller Kalman gains. When Q =, Σ ij =, i.e., no correlation when no process noise. E. Özan Target Tracing December 7, 4 3 / 53 E. Özan Target Tracing December 7, 4 4 / 53 Correlation Illustration Trac Association: Testing Maneuvers mae this problem more dominant and visible. y (m) σa =.3m/s x (m) 8 σa =.3m/s Position Errors (m) time (s) Test for Trac Association [Bar-Shalom (995)]: Two estimates ˆx i, ˆxj and the covariances Σi, Σj from ith and jth local systems. We calculate the difference vector ij Then we calculate covariance Γ ij ij ˆxi ˆxj E( ij ijt ) as Γ ij = Σi + Σj Σij ΣijT are given y (m) x (m) Position Errors (m) time (s) Then test statistics D ij D ij calculated as = ijt (Γ ij ) ij γij can be used for checing trac association. E. Özan Target Tracing December 7, 4 5 / 53 E. Özan Target Tracing December 7, 4 6 / 53

5 Trac Association Trac Association: Assignment Problem What about the cross covariance Σ ij? Simple method is to set it Σ ij =. It can be calculated using Kalman gains if they are transmitted to the fusion center. Approximation for cross covariance from [Bar-Shalom (995)]: The following cross-covariance approximation was proposed: Σ ij ρ (Σ i. Σj where multiplication and power operations are to be done element-wise. For negative numbers, square root must be taen on the absolute value and sign must be ept. The value of ρ must be adjusted experimentally. ρ =.4 was suggested for D tracing. ). Method proposed by [Blacman (999)] for two local agents Form the assignment matrix: A ij T j T j T j 3 T j 4 NA NA NA 3 T i l l l 3 l 4 log β i NA T i l l l 3 l 4 log β i NA T i 3 l 3 l 3 l 33 l 34 log β i NA where l mn log β T P i j P j i N (ˆx m ˆxn ;,Γmn β j NA ) Then, use auction(a ij ) to get trac association decisions.. E. Özan Target Tracing December 7, 4 7 / 53 E. Özan Target Tracing December 7, 4 8 / 53 Trac Association Trac Fusion: Independence Assumption Trac association for more than two local agents. One way is to solve multi dimensional assignment problem. The simpler way is to do the so-called sequential pairwise trac association. Suppose we have N L local agents whose tracs need to be fused. Then, we order the local agents according to some criteria e.g. accuracy, priority, etc. Once we associate two tracs, we have to fuse them to obtain a fused trac. This is called as trac fusion. Consider the trac fusion at point A assuming t CR = t CT = t. Independence assumption gives (Σ A t ) =(Σ B t ) + (Σ C t ) Tracs of Local Agent Tracs of Local Agent Trac Association & Trac Fusion Tracs of Local Agent 3 Trac Association & Trac Fusion Tracs of Local Agent N L Trac Association & Trac Fusion Final Fused Tracs (Σ A t ) ˆx A t =(Σ B t ) ˆx B t + (Σ C t ) ˆx C t This is simplistic and expected to give very bad results here. This is also called as naive fusion. E. Özan Target Tracing December 7, 4 9 / 53 E. Özan Target Tracing December 7, 4 / 53

6 Trac Fusion Extended Target Tracing Illustration of Correlation Independent Schemes. z T (Σ B t ) z = z T (Σ C t ) z = Single extended target modeling Target state: inematics and shape Number of measurements per target Motion modeling not covered. Typically possible to use point target models. z T (Σ A t ) z = Largest Ellipsoid Algorithm Covariance Intersection Multiple extended target tracing Data association for extended targets Measurement set partitioning E. Özan Target Tracing December 7, 4 / 53 E. Özan Target Tracing December 7, 4 / 53 Extended Target Tracing Extended Target Tracing One measurement per target is often not valid, e.g., laser sensors, camera images, or automotive radar. 8 Multiple measurements per target possible to estimate the target s extension. The spatial extension, i.e. size and shape. Hence the name extended target This leads us to the following definition: Extended targets are targets that potentially give rise to more than one measurement per time step. The extension estimate provides additional information, e.g. for classification and identification of different target types: Human? Bie? Car? etc... E. Özan Target Tracing December 7, 4 3 / 53 E. Özan Target Tracing December 7, 4 4 / 53

7 Group targets Closely related to extended targets is group targets. Closely spaced point targets Cannot be resolved Treated as single group target Presented extended target models applicable to group targets. Single extended target Airplane traced with radar Reflection points, or sources, spread across surface. Locations and number vary with time. Measurements z may fall outside target due to noise Extended target state ξ contains parameters for Kinematics Extension Not interested in sources, want to estimate the state ξ given the sets of measurements ( p ξ Z ), Z = { z,..., } zm, Z = {Z,..., Z } E. Özan Target Tracing December 7, 4 5 / 53 E. Özan Target Tracing December 7, 4 6 / 53 Target shape single ellipse Approximate shape as a single ellipse Vector x and positive semi-definite shape matrix X, (z x ) T X (z x ) Target shape single ellipse Extended Object and Group Tracing with Elliptic Random Hypersurface Models, Baum et al. Cholesy decomposition of shape matrix X = L L T [ ] a L = b c Extended state vector ξ = [ x T a ] T b c Assumed Gaussian models p (ξ Z ) =N ( ) ξ ; m, P p (z ξ ) =N (z ; h (ξ ), R ) E. Özan Target Tracing December 7, 4 7 / 53 E. Özan Target Tracing December 7, 4 8 / 53

8 Target shape single ellipse Target shape single ellipse Extended Object and Group Tracing with Elliptic Random Hypersurface Models, Baum et al. Simulation results (plots copied from paper) Bayesian Approach to Extended Object and Cluster Tracing using Random Matrices, Koch. State ξ is combination of inematical vector x and extension matrix X, ξ = (x, X ). Gaussian inverse Wishart distributed, p (ξ Z ) = N ( ) ( ) x ; m, P X IW X ; v, V Called random matrix model Measurement model p (z ξ ) = N (z ; H x, X ), i.e. the covariance is given by the extension matrix. E. Özan Target Tracing December 7, 4 9 / 53 E. Özan Target Tracing December 7, 4 3 / 53 Target shape single ellipse A simple random matrix model. Bayesian Approach to Extended Object and Cluster Tracing using Random Matrices, Koch. Pros: Linear measurement update equations for extension state. Cons: Difficult to model sensor noise. The measurement model p (z ξ ) = N (z ; H x, X ) implies that sensor noise is much smaller than extension. Tracing of extended objects and group targets using random matrices, Feldmann et al. Alternative measurement model p (z ξ ) = N (z ; H x, sx + R ) State ξ is combination of inematical vector x and extension matrix X, ξ = (x, X ). Gaussian inverse Wishart distributed, p (ξ Z ) = N ( ) ( ) x ; ˆx, P IW X ; v, V where, IW (X ; v, V ) = Measurement model v d d V v d Γ d ( v d ) X v p (z ξ ) = N (z ; H x, X ), i.e. the covariance is given by the extension matrix. exp tr( X V ) E. Özan Target Tracing December 7, 4 3 / 53 E. Özan Target Tracing December 7, 4 3 / 53

9 A simple random matrix model. A simple random matrix model. Prediction Update ˆx + = Aˆx, P + = AP A T + BQB, v + = λv, V + = λv, where.5 < λ < is the forgetting factor. Measurement model p (z ξ ) = N (z ; H x, X ), Measurement Update ( p x, X Z ) p(z x, X )p (x, X Z ) Inverse Wishart defines a conjugate prior for the Gaussian lielihood. A family of prior distributions is conjugate to a particular lielihood function if the posterior distribution belongs to the same family as the prior. Hence the posterior is in the same form. i.e. the covariance is given by the extension matrix. E. Özan Target Tracing December 7, 4 33 / 53 E. Özan Target Tracing December 7, 4 34 / 53 A simple random matrix model. Measurement Update Kinematic State where, Extent State ˆx = ˆx + K ( z Cˆx ), P = P K S K T, S = CP C T + ˆX m, K = P C T S v = v + m, V = V + Z, E. Özan Target Tracing December 7, 4 35 / 53 A simple random matrix model. where, z = m z i m, m i= Z = (z i z )(z i z ) T, ˆX = i= V v d Tracing of extended objects and group targets using random matrices, Feldmann et al. State of the art filter p (z ξ ) = N (z ; H x, sx + R ) Better sensor noise model, but the measurement update is heuristic and no longer linear. E. Özan Target Tracing December 7, 4 36 / 53

10 Target shape single ellipse Target shape multiple ellipses Random matrix model popular in the literature, see e.g. Probabilistic tracing of multiple extended targets using random matrices, Wienee and Koch. A PHD filter for tracing multiple extended targets using random matrices, Granström and Orguner. Tracing of Extended Object or Target Group Using Random Matrix Part I: New Model and Approach, Lan and Rong Li. and the reference therein. A single ellipse can be a crude model, multiple ellipses gives better approximation, see e.g. Arbitrary number of ellipses, not same center point. Tracing of Extended Object or Target Group Using Random Matrix Part II: Irregular Object, Lan and Rong Li. Single ellipse also used by Degerman et al in Extended Target Tracing using Principal Components. The shape matrix decomposed into rotation, major axis, and minor axis. E. Özan Target Tracing December 7, 4 37 / 53 E. Özan Target Tracing December 7, 4 38 / 53 Taregt shape Star convex A set S(x ) is called star-convex if each line segment from the center to any point is fully contained in S(x ). y-coordinate Radius r r θ x-coordinate r = f(θ) π π Angle θ E. Özan Target Tracing December 7, 4 39 / 53 Figure: Description of star-convex shapes using a radius function r = f(θ). Target shape general shapes Single, or multiple, ellipses can be a crude model of the shape. Need for more general shape model. Shape Tracing of Extended Objects and Group Targets with Star-Convex RHMs, Baum and Hanebec. The shape is given by a radial function r(θ) distance from center of target to edge of shape at angle θ. Circle: r = f(θ) Canonical ellipse: r(θ) = a cos(θ) + b sin(θ) General case: truncated Fourier series expansion r(θ) = a NF + a j cos(jθ) + b j sin(jθ) j= Shape parameter: p = [a, a,..., a NF, b,..., b NF ] T E. Özan Target Tracing December 7, 4 4 / 53

11 Target shape general shapes Multiple extended target tracing Shape Tracing of Extended Objects and Group Targets with Star-Convex RHMs, Baum and Hanebec. Compare with ellipse (plots copied from paper): Assume that we have chosen the random matrix model for the extended targets. How can we handle multiple extended targets, when there is clutter and detection uncertainty? As with point target tracing, the data association problem must be solved. Different because each target can generate multiple measurements. Which measurements were caused by clutter sources? Which measurements were caused by targets? Of those, which were caused by the same target? E. Özan Target Tracing December 7, 4 4 / 53 E. Özan Target Tracing December 7, 4 4 / 53 Data association for extended targets Partitioning the measurements example { Partition the measurement set Z = z (), z(), z(3) }.5 Z.5 p.5 p One approach: Partitioning of the measurement set y.5 z () z () y.5 W y.5 W A partition p is a division of the set Z into non-empty subsets, called cells W. z (3) x x W x Interpretation: One cell = one source. y.5.5 W 3 p 3 W 3 y.5.5 W 4 p 4 W 4 y.5.5 p 5 W 5 W 5 W x x x E. Özan Target Tracing December 7, 4 43 / 53 E. Özan Target Tracing December 7, 4 44 / 53

12 Partitioning method Partitioning method laser example For optimality we must consider all possible partitions. Number of possible partitions for n measurements given by n:th Bell number B n. Intuition: Measurements are from same source if they are close, with respect to some measure or distance. The Bell numbers B n increase very fast when n increases, e.g. B 3 = 5, B 5 = 5 and B = Computationally infeasible to consider all partitions. Necessary to approximate the full set of partitions with a subset of partitions E. Özan Target Tracing December 7, 4 45 / 53 E. Özan Target Tracing December 7, 4 46 / 53 Partitioning method laser example Partitioning method Intuition: Measurements are from same source if they are close, with respect to some measure or distance. 8 Method: Measurements are in same cell W if distance is small. Distance between two cells W j and W i is measured as min {d (z i, z j ) : z i W i, z j W j } Partitions p i where all pairwise cell distances is < d i. Limit to partitions for thresholds d i that satisfy d min d i < d max If possible, use scenario nowledge to choose distance measure and to determine bounds. Important to choose d min and d max conservatively. E. Özan Target Tracing December 7, 4 46 / 53 E. Özan Target Tracing December 7, 4 47 / 53

13 Partitioning example p = { W, W, W 3 } p = { W, W, W 3, W 4 } Extended Target Models Extended target models applicable in scenarios where there are multiple measurements per target Also applicable when multiple targets move in tight groups Shape models range from simple ellipse to general shapes. Reasonable to discard most partitions as highly unliely. Additional methods given in: Extended Target Tracing using a Gaussian-Mixture PHD filter, Granström et al. A PHD filter for tracing multiple extended targets using random matrices, Granström and Orguner Multiple extended target tracing possible using, e.g., the extended target PHD filter. E. Özan Target Tracing December 7, 4 48 / 53 E. Özan Target Tracing December 7, 4 49 / 53 Extended Multi-target Tracing- Recent Wors Extended Target Examples A PHD filter for tracing multiple extended targets using random matrices, Granström and Orguner. Laser Pedestrian Video Pedestrian Laser - Mix 3 A Multiple-detection joint probabilistic data association filter, Habtemariam et al. 5 A multiple hypothesis tracer for multitarget tracing with multiple simultaneous measurements, Sathyan et al. y [m] 5 4 y [m] x [m] 5 5 x [m] E. Özan Target Tracing December 7, 4 5 / 53 E. Özan Target Tracing December 7, 4 5 / 53

14 References References M. E. Liggins II, Chee-Yee Chong, I. Kadar, M. G. Alford, V. Vannicola and V. Thomopoulos, Distributed fusion architectures and algorithms for target tracing, Proceedings of the IEEE, vol.85, no., pp. 95-7, Jan M. E. Liggins and Kuo-Chu Chang, Distributed fusion architectures, algorithms, and performance within a networ-centric architecture, Ch.7, Handboo of Multisensor Data Fusion: Theory and Practice, Taylor & Francis, Second Edition, 9. K. C. Chang, Chee-Yee Chong and S. Mori, On scalable distributed sensor fusion, Proceedings of th International Conference on Information Fusion, Jul. 8. S. Blacman and R. Popoli, Design and Analysis of Modern Tracing Systems. Norwood, MA: Artech House, 999. Y. Bar-Shalom and X. R. Li, Multitarget-Multisensor Tracing: Principles, Techniques. Storrs, CT: YBS Publishing, 995. Granström et al., Extended Target Tracing using a Gaussian-Mixture PHD filter Granström and Orguner, A PHD filter for tracing multiple extended targets using random matrices E. Özan Target Tracing December 7, 4 5 / 53 E. Özan Target Tracing December 7, 4 53 / 53