Particle Filters for Visual Tracking

Transcription

1 Particle Filters for Visual Tracking T. Chateau, Pascal Institute, Clermont-Ferrand 1

2 Content Particle filtering: a probabilistic framework SIR particle filter MCMC particle filter RJMCMC particle filter Particle Filters for Visual Tracking T. Chateau 2

3 Content Particle filtering: a probabilistic framework SIR particle filter MCMC particle filter RJMCMC particle filter Particle Filters for Visual Tracking T. Chateau 3

4 Visual Tracking State (hidden variable) y bird view x Observation y x Particle Filters for Visual Tracking T. Chateau 4

5 Definitions Dynamic Bayesian Network representation x k 3 x k 2 x k 1 x k x k+1 x k+2 z k 3 z k 2 z k 1 z k z k+1 z k+2 States X. = {x k } k=1,...,k Observations Z. = {z k } k=1,...,k Particle Filters for Visual Tracking T. Chateau 5

6 Online Tracking Online Tracking Observation z k 3 z k 2 z k 1 z k z k+1 z k+2 State x k 1 x k Particle Filters for Visual Tracking T. Chateau 6

7 Online Tracking Probabilistic framework p(z k X k ) Observation z k 3 z k 2 z k 1 z k z k+1 z k+2 State x k 1 x k p(x k 1 Z 0:k 1 ) p(x k Z 0:k ) Particle Filters for Visual Tracking T. Chateau 7

8 Sequential Monte-Carlo Inference p(x k 1 z k 1 ) Prediction (Chapman Kolmogorov) p(x k z k 1 ) Update (Bayes) p(x k z k ) p(x k x k 1 ) p(z k x k ) Dynamics Likelihood Particle Filters for Visual Tracking T. Chateau 8

9 Sequential Monte-Carlo Inference p(x k 1 z k 1 ) Prediction (Chapman Kolmogorov) p(x k z k 1 ) Update (Bayes) p(x k z k ) p(x k x k 1 ) p(z k x k ) Dynamics Likelihood Stochastic representation of distributions Particle Filters for Visual Tracking T. Chateau 9

10 Stochastic representation of distributions With a set of particles p(x) 1 N NX n=1 (X X n ) With a set of weighted particles p(x) NX n (X X n ) n=1 Particle Filters for Visual Tracking T. Chateau 10

11 Visual Tracking State (hidden variable) y bird view x Observation y x Particle Filters for Visual Tracking T. Chateau 11

12 Content Particle filtering: a probabilistic framework SIR particle filter MCMC particle filter RJMCMC particle filter Particle Filters for Visual Tracking T. Chateau 12

13 SIR Particle Filter Sampling Importance Resampling y x State distribution at time t-1 Resampling y x Particle Filters for Visual Tracking T. Chateau 13

14 SIR Particle Filter State distribution at time t-1 Prediction Particle Filters for Visual Tracking T. Chateau 14

15 SIR Particle Filter Predicted distribution at time t Update Posterior S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for on-line non-linear/ non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50(2): , Feb Particle Filters for Visual Tracking T. Chateau 15

16 SIR Particle Filter t=1 t=2 t t=3 Target Temporal Filtering t=4 Particle Filters for Visual Tracking T. Chateau 16

17 SIR Particle Filter: some examples State vector: 2D position and scale (image reference frame) Dynamics: random step Observation model: max. of gradients set of points T. Chateau and J. Lapresté. Robust real time tracking of a vehicle by image processing. In IEEE Intelligent Vehicles Symposium,, Parma, Italy, June Particle Filters for Visual Tracking T. Chateau 17

18 SIR Particle Filter: some examples State vector: 2D position and scale (image reference frame) Dynamics: random step Observation model: offline learning based model (Haar wavelets) T. Chateau, V. Gay-Belille, F. Chausse, and J. T. Lapresté. Real-time tracking with classifiers. In WDV - WDV Workshop on Dynamical Vision at ECCV2006, Grazz, Austria, May Particle Filters for Visual Tracking T. Chateau 18

19 SIR Particle Filter: some examples State vector: 3D position, orientation, steering angle and velocity Dynamics: driven by a bicycle model Observation model: background/foreground subtraction, camera and laser range finder Collaboration with LCPC Nantes Y. Goyat, T. Chateau, and L. Trassoudaine. Tracking of vehicle trajectory by combining a camera and a laser rangefinder. Springer MVA : Machine Vision and Application, online, March Particle Filters for Visual Tracking T. Chateau 19

20 SIR Particle Filter: some examples Particle Filters for Visual Tracking T. Chateau 20

21 Efficient implementation of SIR Particle Filters Particle Filters for Visual Tracking T. Chateau 21

22 Efficient implementation of Particle Filters SIR Particle Filter N / e d N: number of particles d : size of the state vector Particle Filters for Visual Tracking T. Chateau 22

23 Content Particle filtering: a probabilistic framework SIR particle filter MCMC particle filter RJMCMC particle filter Particle Filters for Visual Tracking T. Chateau 23

24 MCMC Particle Filter p(x k 1 Z k 1 ) Particle Filters for Visual Tracking T. Chateau 24

25 MCMC Particle Filter p(x k 1 Z k 1 ) Draw Particle Filters for Visual Tracking T. Chateau 24

26 MCMC Particle Filter p(x k 1 Z k 1 ) Draw Move X 0 Particle Filters for Visual Tracking T. Chateau 24

27 MCMC Particle Filter Markov Chain Monte Carlo X 0 Particle Filters for Visual Tracking T. Chateau 25

28 MCMC Particle Filter Markov Chain Monte Carlo Draw X 0 Particle Filters for Visual Tracking T. Chateau 25

29 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0 Particle Filters for Visual Tracking T. Chateau 25

30 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0 Particle Filters for Visual Tracking T. Chateau 25

31 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0 Particle Filters for Visual Tracking T. Chateau 25

32 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0? Compare Particle Filters for Visual Tracking T. Chateau 25

33 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0? Compare X 1 Accept X or copy X 0 Particle Filters for Visual Tracking T. Chateau 25

34 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0? Compare X 1 X N Accept X or copy X 0 Particle Filters for Visual Tracking T. Chateau 25

35 MCMC Particle Filter Markov Chain Monte Carlo X Move Draw X 0? Compare X 1 X N Accept X or copy X 0 Particle Filters for Visual Tracking T. Chateau 25

36 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

37 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

38 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

39 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

40 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

41 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

42 MCMC Particle Filter: marginal move Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

43 MCMC Particle Filter: marginal move! =min 1,... t k... Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

44 MCMC Particle Filter: marginal move! =min 1,... t k... Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

45 MCMC Particle Filter: marginal move! =min 1,... t k... Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

46 MCMC Particle Filter: marginal move! =min 1,... t k... Metropolis Hasting rule Particle Filters for Visual Tracking T. Chateau 26

47 MCMC Particle Filter: example - State vector: 2D location, scale and combination parameters of observation modules (colour, texture, gradient,...) - Dynamics: random step - Observation function: learning based (Adaboost) - In collaboration with Teb-online Particle Filters for Visual Tracking T. Chateau 27

48 RJMCMC Particle Filter Used to track a varying number of objects X t. = {It, x 1 t,..., x I t t } The state is defined into a joint space Particle Filters for Visual Tracking T. Chateau 28

49 RJMCMC Particle Filter Reversible Jump Markov Chain Monte Carlo Approximate the distribution with a variable size A pair of new proposals X = {x 1,x 2,x 3 } X = {x 1,x 2 } Jump into a lower dimensional space X = {x 1,x 2,x 3,x 4 } Jump into a higher dimensional space Particle Filters for Visual Tracking T. Chateau 29

50 RJMCMC Particle Filter Proposals have to be add: - object position update - add one object - remove one object The state is defined into a joint space Data driven Particle Filters for Visual Tracking T. Chateau 30

51 RJMCMC Particle Filter Add an object: a data driven proposal Background/foreground hypothesis X New object position proposal map Background/foreground observation map Particle Filters for Visual Tracking T. Chateau 31

52 RJMCMC Particle Filter Real time pedestrian tracking Particle Filters for Visual Tracking T. Chateau 32

53 RJMCMC Particle Filter Real time vehicle tracking Particle Filters for Visual Tracking T. Chateau 33

54 Proposals: - update object position - add/remove one object - update object category RJMCMC Particle Filter Simultaneous tracking and categorisation One geometric and kinematic model for each category Particle Filters for Visual Tracking T. Chateau 34

55 Proposals: - update object position - add/remove one object - update object category RJMCMC Particle Filter Simultaneous tracking and categorisation One geometric and kinematic model for each category Particle Filters for Visual Tracking T. Chateau 34

56 Proposals: RJMCMC Particle Filter Simultaneous tracking and categorisation - update object position - add/remove one object - update object category category update proposal matrix Particle Filters for Visual Tracking T. Chateau 35

57 RJMCMC Particle Filter Simultaneous tracking and categorisation F. Bardet, T. Chateau, and D. Ramadasan. Unifying real-time multi-vehicle tracking and categorization. In Intelligent Vehicle Symposium, volume 1, Particle Filters for Visual Tracking T. Chateau 36

58 RJMCMC Particle Filter Simultaneous tracking, categorisation and context detection Proposals: - update object or sun position position - add/remove one object or sun - update object category Particle Filters for Visual Tracking T. Chateau 37

59 RJMCMC Particle Filter Simultaneous tracking and categorisation F. Bardet, T. Chateau, and J. Lapresté. Illumination aware mcmc particle filter for long-term outdoor multi-object simultaneous tracking and classification. In ICCV 2009, International Conference on Computer Vision, Tokyo, Japan, Particle Filters for Visual Tracking T. Chateau 38

60 RJMCMC Particle Filter Particle Filters for Visual Tracking T. Chateau 39

61 Efficient implementation of MCMC Particle Filters Multi Proposal MCMC Particle Filter X n 1 1 α π n 1 α X π X n π n X n 1 1 α π n 1 X1 π1 XP πp α X n π n X p π p p {1,..., P } (a): single proposal MCMC (b): P-proposal MCMC Particle Filters for Visual Tracking T. Chateau 40

62 t = MCMC 1 PF Efficient implementation of Particle Filters MCMC 2 PF MCMC 4 PF t = t = t = t = Particle Filters for Visual Tracking T. Chateau 41

63 Conclusion Particle filters are widely used for temporal filtering applications They provide tools able to handle with non linear systems SIR particle filters have to be chosen for low dimensional problems MCMC particle filters with marginal proposal strategy are preferred for high dimensional problems RJMCMC particle filters can be used to manage states with a varying dimension Particle Filters for Visual Tracking T. Chateau 42