Tracking wrapup Course recap. Announcements

Transcription

1 Tracking wrapup Course recap Tuesda, Dec 1 Announcements Pset 4 grades and solutions available toda Reminder: Pset 5 due 12/4, extended to 12/8 if needed Choose between Section I (short answers) and II (program) Extra credit onl given for Section III Final exam is 12/14 Monda Toda s handout has example final exams Thursda in class: exam review 1

2 Previousl Tracking as inference Goal: estimate posterior of object position given measurement Linear models of dnamics Represent state evolution and measurement models Kalman filters Recursive prediction/correction updates to refine measurement General tracking challenges Last time: Tracking as inference The hidden state consists of the true parameters we care about, denoted X. The measurement is our nois observation that results from the underling state, denoted Y. At each time step, state changes (from X t-1 to X t ) and we get a new observation Y t. Our goal: recover most likel state X t given All observations seen so far. Knowledge about dnamics of state transitions. 2

3 Last time: Tracking as inference measurement Belief: prediction Belief: prediction Corrected prediction old belief Time t Time t+1 Last time: Linear dnamic model Describe the a priori knowledge about Sstem dnamics model: represents evolution of state t over time, with noise. xt ~ N( Dxt 1; Σd ) Measurement model: at ever time step we get a nois measurement of the state. t ~ N( Mxt; Σm) 3

4 Last time: Kalman filter Know corrected state from previous time step, and all measurements up to the current one Predict distribution over next state. Time update ( Predict ) P ( X K ) t 0,, t 1 Receive measurement Know prediction of state, and next measurement Update distribution over current state. Measurement update ( Correct ) ( X, ) P, K t 0 t Mean and std. dev. of predicted state: μ t, σ t Time advances: t++ Mean and std. dev. of corrected state: + + μ t, σ t Kalman filter: pros and cons Gaussian densities, linear dnamic model: + Simple updates, compact and efficient But, restricted class of motions defined b linear model Unimodal distribution = onl single hpothesis x ~ N (μ, μ Σ ) 4

5 When is a single hpothesis too limiting? initial position prediction measurement update x x x x Figure from Thrun & Kosecka When is a single hpothesis too limiting? initial position prediction measurement update x x x x Consider this example: sa we are tracking the face on the right using a skin color blob to get our measurement. Video from Jojic & Fre 5

6 When is a single hpothesis too limiting? initial position prediction measurement update x x x x Consider this example: sa we are tracking the face on the right using a skin color blob to get our measurement. Video from Jojic & Fre Alternative: particle-filtering and non-gaussian densities Can represent distribution with set of weighted samples ( particles ) Allows us to maintain multiple hpotheses. For details: CONDENSATION -- conditional densit propagation for visual tracking, b Michael Isard and Andrew Blake, Int. J. Computer Vision, 29, 1, 5--28, (1998) 6

7 Alternative: particle-filtering and non-gaussian densities Monitor is a distractor, t multiple l Kalman filter fails once it starts t hpotheses necessar. tracking the monitor. Visual Dnamics Group, Dept. Engineering Science, Universit of Oxford, 1998 Tracking people b learning their appearance Tracker D. Ramanan, D. Forsth, and A. Zisserman. Tracking People b Learning their Appearance. PAMI Source: Lana Lazebnik 7

8 Tracking people b learning their appearance Use a part-based model to encode part appearance + relative geometr. Bottom-up initialization: Clustering D. Ramanan, D. Forsth, and A. Zisserman. Tracking People b Learning their Appearance. PAMI Source: Lana Lazebnik 8

9 Top-down initialization: Exploit eas poses D. Ramanan, D. Forsth, and A. Zisserman. Tracking People b Learning their Appearance. PAMI Tracking b model detection D. Ramanan, D. Forsth, and A. Zisserman. Tracking People b Learning their Appearance. PAMI

10 Example results Example results 10

11 Example results Example results 11

12 Tracking : summar Tracking as inference Goal: estimate posterior of object position given measurement Linear models of dnamics Represent state evolution and measurement models Kalman filters Recursive prediction/correction updates to refine measurement Single hpothesis can be limiting General tracking challenges Tracking via detection one wa to mitigate drift (though means losing out on prediction help). Course recap 12

13 Features and filters Transforming and describing images; textures, colors, edges Grouping & fitting Clustering, segmentation, fitting; what parts belong together? [fig from Shi et al] 13

14 Multiple views Multi-view geometr, matching, invariant features, stereo vision Lowe Hartle and Zisserman Fei-Fei Li Recognition and learning R i i bj t Recognizing objects and categories, learning techniques 14

15 Motion and tracking Tracking objects, video analsis, low level motion, optical flow Tomas Izo Computer Vision Automatic understanding of images and video 1 Computing properties of the 3D world from visual 1. Computing properties of the 3D world from visual data (measurement) 15

16 1. Vision for measurement Real-time stereo Structure from motion Tracking NASA Mars Rover Snavel et al. Demirdjian et al. Wang et al. Computer Vision Automatic understanding of images and video 1. Computing properties of the 3D world from visual data (measurement) 2. Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation) 16

17 2. Vision for perception, interpretation The Wicked Twister ride Lake Erie sk water Ferris wheel amusement park Cedar Point tree ride 12 E Objects Activities Scenes Locations Text / writing Faces Gestures Motions Emotions ride tree people waiting in line people sitting on ride deck tree bench tree umbrellas carousel pedestrians maxair Computer Vision Automatic understanding of images and video 1. Computing properties of the 3D world from visual data (measurement) 2. Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation) 3. Algorithms to mine, search, and interact with visual data (search and organization) 17

18 3. Visual search, organization Quer Image or video archives Relevant content Visual data in 1963 L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering,

19 Visual data in 2009 Personal photo albums Movies, news, sports Surveillance and securit Medical and scientific images Slide credit; L. Lazebnik Wh vision? As image sources multipl, so do applications Relieve humans of boring, eas tasks Enhance human abilities Advance human-computer interaction, visualization Perception for robotics / autonomous agents Organize and give access to visual content 19

20 Faces and digital cameras Camera waits for everone to smile to take a photo [Canon] Setting camera focus via face detection Linking to info with a mobile device Situated search Yeh et al., MIT kooaba MSR Lincoln 20

21 Video-based interfaces Human jostick NewsBreaker Live Assistive technolog sstems Camera Mouse Boston College Vision for medical & neuroimages fmri data Golland et al. Image guided surger MIT AI Vision Group 21

22 Special visual effects The Matrix What Dreams Ma Come Mocap for Pirates of the Carribean, Industrial Light and Magic Source: S. Seitz Safet & securit Navigation, driver safet Monitoring pool (Poseidon) Pedestrian detection MERL, Viola et al. Surveillance 22