Visual Perception as Bayesian Inference. David J Fleet. University of Toronto

Transcription

1 Visual Percepion as Bayesian Inference David J Flee Universiy of Torono

2 Basic rules of probabiliy sum rule (for muually exclusive a ): produc rule (condiioning): independence (def n ): Bayes rule: marginalizaion:

3 Inference & Bayes Rule Model Parameers: Daa (Observaions): Poserior Likelihood Prior

4 Visual racking Inference of objec shape, appearance, and moion from video.

5 Why is racking so hard? complex nonlinear dynamics, wih high dimensional objec models complex appearance, and emporal appearance variaion (deformable objecs, shadows & lighing variaions, clohing, ) ambiguiy: occlusion, scale, background cluer, muliple objecs, uncerain models, and weakly consrained parameers

6 Probabilisic formulaion Objec model is a se of variables (properies of he image or scene) ha we wish o esimae as a funcion of ime? Sae: n-vecor conaining variables o be esimaed: - coninuous variables [eg., posiion, velociy, shape, size, ] - discree sae variables [eg., # objecs, gender, aciviy, ] - sae hisory: Observaions: daa from which we esimae sae: - observaion hisory:

7 Probabilisic formulaion Poserior disribuion over saes condiioned on observaions Filering disribuion: marginal poserior a curren ime Bayes rule: likelihood prior independen of sae

8 Model assumpions 1 s -order Markov model for sae dynamics: so sequence one-sep prior ransiion prior Condiional independence of observaions join likelihood likelihood a ime τ

9 Filering equaions Filering disribuion: likelihood predicion Predicion disribuion (emporal prior): This provides a recursive expression for he poserior.

10 Recursive filering deerminisic drif poserior incorporae daa sochasic diffusion poserior predicion

11 Kalman filer Assume lineariy & Gaussianiy for observaion and dynamics eqns: Key Resul: Key Resul: Predicion and filering disribuions are Gaussian, so hey may be represened by sufficien saisics:

12 Kalman filer Firs well-known uses in compuer vision: Road following by racking lane markers [Dickmanns & Graefe, Dynamic monocular machine vision. Machine Vision and Applicaions, 1988] Rigid srucure from feaure racks under perspecive projecion [Broida e al., Recursive esimaion of 3D moion from monocular image sequence. IEEE Trans. Aerosp. & Elec. Sys., 1990]

13 E.g., Vehicle racking [Koller, Weber & Malik, Robus muliple car racking wih occlusion reasoning. Proc ECCV,1994]

14 Problems remain: Muli-modal likelihoods Measuremen cluer and occlusion in naural images ofen cause likelihood funcions o have muliple, local maxima.

15 Ambiguiy & muliple objecs [Birchfield, Ellipical head racking using inensiy gradiens and color hisograms. Proc CVPR, 1998]

16 Problems Remain: Non-Linear Dynamics Animae objecs and he ineracions beween objecs ofen produce complex nonlinear dynamics Non-linear dynamics do no preserve simple disribuions [Jepson, Flee and El-Maraghi, WSL Tracker, IEEE Trans. PAMI, 2001]

17 Bayesian filering deerminisic drif poserior incorporae daa sochasic diffusion poserior predicion

18 Non-parameric approximae inference Approximae he filering disribuion using poin samples: By drawing a se of random samples from he filering disribuion, we could use samples saisics o approximae expecaions Le be a se of fair samples from disribuion, hen for funcions Problem: we don know how o draw samples from

19 Imporance sampling weighed samples Weighed sample se draw samples from a proposal disribuion, wih weighs, hen If hen weighed sample saisics approximae expecaions under, i.e.,

20 Paricle filers Sequenial Mone Carlo mehods draw weighed samples o approximae he filering disribuion: Simple paricle filer (wih resampling a each ime sep): draw samples from he predicion disribuion weighs are proporional o he raio of poserior and predicion disribuions, i.e. he normalized likelihood sample sample normalize poserior emporal dynamics likelihood poserior [Gordon e al 93; Isard & Blake 98; Liu & Chen 98, ]

21 Sampling he predicion disribuion Given a weighed sample se predicion disribuion is a linear mixure model, he To draw a sample from i: - sample a componen of he mixure by he reaing weighs as mixing probabiliies Cumulaive disribuion of weighs sample N - hen sample from he associaed dynamics pdf

22 Paricle filers weighed sample se re-sample & drif diffuse & re-sample compue likelihoods weighed sample se [Isard and Blake, 98]

23 2D Conour Tracking Sae: 6 parameers of affine deformaion. Measuremens: edge srengh perpendicular o conour Dynamics: 2 nd -order Markov model (ofen learned) [Isard & Blake, Condensaion - condiional densiy propagaion for visual racking. IJCV, 1998]

24 2D conour racking (6 DOF sae space, 1000 paricles)

25 2D conour racking (6D affine sae, 100 paricles) (6D affine sae, 1200 paricles) [Isard & Blake, Condensaion - condiional densiy propagaion for visual racking. IJCV, 1998]

26 2.1D blob racking Sae: number of people, heir posiions/velociies on ground plane, and simple shape models (10 dimensions / person) Appearance: filer response hisograms for background, and for people Dynamics: damped 2 nd -order model for posiion/velociy, 1 s -order for shape model (1 person required ~500 paricles, 2-3 people required >10,000 paricles) [Isard and MacCormick, Bramble: A Muliple Blob Bayesian Tracker. Proc ICCV, 2001]

27 Monocular 3D people racking 3D Kinemaic Model (28D sae, wih 22 join angles, 6 global DOFs)

28 Likelihood and dynamics Given he sae,, and he ariculaed model, he 3D marker posiions ono he 2D image plane: Observaion model: Likelihood of observed 2D locaions, : Smooh dynamics: where is isoropic Gaussian for ranslaional & angular variables

29 Experimenal evaluaion Esimaor Variance: muliple runs wih independen noise & sampling variance measured as MSE from ground ruh (from MCMC)

30 Experimenal evaluaion Black: ground ruh (a frame 10) Red: mean saes from 6 random rials

31 Problem: Exponenial numbers of samples? Number of samples needed depends on he effecive volumes (enropies) of he predicion and poserior disribuions. wih ramdom sampling from he predicion densiy, he number of paricles mus grow exponenially in sae dimension for samples o fall saes wih high poserior E.g., for D-dim spheres, wih radii R and r, Predicion Poserior effecive number of independen samples:

32 Hybrid Mone Carlo filer Paricles can exploi informaion obained by exising paricles iniialize a se of paricles from a paricle filer selec a subse from which o begin MCMC simulaion wih sochasic gradien ascen (hybrid Mone Carlo) opimisic exrapolan fixed variance HMC filer >> 2,000 faser HMC filer paricle filer [Choo & Flee, People Tracking Using Hybrid Mone Carlo Filering., Proc IEEE ICCV, 2001]

33 Experimenal resuls Paricle Filer Hybrid MC Filer Black: Ground ruh (a frame 10) Red: Mean sae from 6 random rials

34 Learning aciviy-specific dynamics Walking Model: Join angle curves are segmened and scaled o yield daa curves where is he phase he walking cycle. PCA provides a linear basis for he join angles a phase : mean knee angle knee angle basis [Sidenbladh, Black & Flee, Sochasic racking of 3D human figures using 2D image moion. Proc ECCV, 2000]

35 Temporal dynamics: Walking model mean walking mean walking plus moderae noise mean walking plus large noise

36 Temporal dynamics: Walking model Smooh emporal dynamics (Gaussian process noise): ), ( ) (, 1, 1,, c k k k k k c c G c c p σ = ), ( ), ( τ σ τ τ τ τ g g g g g g v G v p = ), ( ) ( ψ σ ψ ψ ψ ψ = v G p ), ( ) ( 1 1 v v v G v v p σ = ), ( ) ( 1 1 θ σ θ θ θ θ g g g g G p = k c, 1 Parameers of he generaive model a ime : 5 basis coefficiens global pose phase speed

37 Moion likelihood 1 A -1 = M( D 1; φ1) D = M 1 ( A1 ; φ ) + η heavy-ailed noise Image formaion: perspecive projecion of exure-mapped 3D shape (assumes brighness consancy and addiive noise)

38 3D people racking Mean poserior sae shown from wo viewpoins. (15000 paricles, manual iniializaion) [Sidenbladh, Black & Flee, Sochasic racking of 3D human figures using 2D image moion. Proc ECCV, 2000]

39 Tracking hockey players Adaboos used o rain a 23 layer classifier o deec hockey players: 2000 negaive examples from locaions on rink wihou players 200 posiive examples Key (Haar) feaures: [Okuma e al., Boosed Paricle Filer. Proc. ECCV 2004]

40 Tracking hockey players Sae: number of players, plus posiions / velociies (in rink coords) Appearance: color hisograms for op & boom Facored Poserior: independen filers applied o players (unless players in close proximiy) [Okuma e al., Boosed Paricle Filer. Proc. ECCV 2004]