Stochastic geometry. automatic object detection and tracking. remotely sensed image sequences

Transcription

1 Stochastic geometry for automatic object detection and tracking in remotely sensed image sequences Paula CR This work has been done in collaboration with dr. Josiane and dr. Mathias Ortner

2 Surveillance now more than ever Human benefits Wildlife benefits 2

3 Optical airborne and spaceborne systems -meter ground sampling resolution imagery Low-orbit satellites -meter ground sampling resolution imagery High- Geostationary satellites Low temporal frequency

4 Challenges object size Large number of objects object motion Time requirements

5 Multiple Object Tracking (MOT) Goal: Extract object trajectories throughout a Two sub-problems Where are the possible targets? - targets Which detection corresponds to each target? - data association problem Two data-handling approaches Sequential order Batch processing once approaches Tracking by detection Track before detect 5

6 Data-association based methods NN-app DD-MCMCDA MHT [ 6

7 RFS-based methods RFS PHD L-RFS / GLMB [ 7

8 Patterns and stochastic geometry Object tracking as a spatio-temporal marked point process How to model and simulate such a spatio-temporal point process?

9 Thesis at a glance Marked point process models for object detection and tracking Linear programming for automatic or semi-automatic parameter learning Model simulation using

10 Overview Models Model formulation Quality model Parameter learning Linear programming Parameter learning as a linear program RJMCMC with Kalman inspired Parallel implementation of RJMCMC Results

11 Marked point process of ellipses Center of the ellipse is a point in the point process Marks: Geometric marks: semi-- : label y 11 x

12 Marked Point Process for Multiple Object Tracking Multiple object tracking problem X image sequence Y X The process energy is composed of two energy terms: 12 External energy Internal energy

13 Internal energy Long smooth trajectories

14 External energy Quality model Statistical model frame differencing Contrast distance measure between interior and exterior of ellipse

15 Total energy Quality model Statistical model

16 Overview Models Model formulation Quality model Parameter learning Linear programming Parameter learning as a linear program RJMCMC with Kalman inspired Parallel implementation of RJMCMC Results

17 Linear programming

18 Objective function Quality model energy formulation function

19 Gathering constraints Only the ratio is needs to be computed We can create inequalities of the form we ground truth information Or more specifically the constraints can be written as

20 How many constraints?

21 Overview Models Model formulation Quality model Parameter learning Linear programming Parameter learning as a linear program RJMCMC with Kalman inspired Parallel implementation of RJMCMC Results

22 Related samplers RJMCMC MBD and MBC P-RJMCMC 22

23 Classic RJMCMC Why? Highly non- energy MCMC number of objects RJ Core idea Create a chain perturb the current state of the chain is reached

24 Standard perturbation kernels Local transformations Rotation Translation

25 RJMCMC sampler X i 1 X i - Xi 1 - X i 2 X i 2 25

26 Adding Kalman-inspired births Perturbation accepted Kalman End iteration Perturbation rejected Perturbation accepted Kalman 26

27 Did time efficiency increase? RJMCMC with Kalman like moves standard RJMCMC Experimental results 27 Kalman-inspired births reduce computation times!

28 Parallel implementation of RJMCMC [Verdie2012] Data-driven space partitioning Locally conditional independent perturbations Image with boats Airbus D&S

29 Parallel implementation of RJMCMC Data-driven space partitioning Locally conditional independent perturbations Probability that objects exist in each part of the image

30 Parallel implementation of RJMCMC Data-driven space partitioning Locally conditional independent perturbations Color coding of quad-tree leafs

31 Parallel perturbations [Verdie2012] A color is randomly chosen Perturbations are performed in all cells of the chosen color in parallel Color blue is randomly chosen

32 Our improvement to the parallel sampler Problem Solution Large boat is split between two neighboring cells Take the configurations in the neighboring cells into consideration

33 Did time efficiency increase? RJMCMC RJMCMC + Kalman Parallel RJMCMC without Kalman Parallel implementation significantly reduces computation times!

34 Overview Models Model formulation Quality model Parameter learning Linear programming Parameter learning as a linear program RJMCMC with Kalman inspired Parallel implementation of RJMCMC Results

35 Data sets 2 different data sets: -2

36 UAV data low temporal frequency COLUMBUS LARGE IMAGE FORMAT Original image Proposed

37 UAV data low temporal frequency Original image Proposed

38 Satellite data low temporal frequency

39 Satellite data high temporal frequency Tracking results

40 Satellite data high temporal frequency -

41 Overview Models Model formulation Quality model Parameter learning Linear programming Parameter learning as a linear program RJMCMC with Kalman inspired Parallel implementation of RJMCMC Results

42 Conclusions Two spatio-temporal marked point process models for the detection and objects -automatic parameter estimation using linear programming RJMCMC sampler with Kalman-like Efficient parallel implementation of the RJMCMC sampler Good results on different types of data

43 Critical analysis Advantages Drawbacks of weakly contrasted objects Consistent trajectories Object interactions modeling Robustness to noise and data quality Good results on different data sets Real-time processing only in exceptional cases

44 Perspectives a hierarchical model that integrates both low- objects and high- constraints between trajectories Multi-marked process to distinguish between classes Model traffic density instead of trajectories Optimization to make

45 References X. Minlos Zhizhina. Object extraction using a stochastic birth-and- Gamal Eldin Charpiat Zerubiabirthand model determination. R. Mahler. Multitarget ltering rst- - Papi - approximation of multi- PereraHoogsHu. Multi-object tracking through simultaneous long occlusions and split- Multiframe many- Modelisation de scènes urbaines à partir de donnèes aeriennes Phung. Labeled random finite sets and the --nite sets and multi- Q. Yu and G. Medioni. Multiple-target tracking by spatio-temporal Monte Carlo chain Trans