Announcements. Motion. Motion. Continuous Motion. Background Subtraction

Transcription

1 Annoncements Motion CSE 5A Lectre 13 Homework is de toda, 11:59 PM Reading: Section : Optical Flow and Motion Section 10.6.: Flow Models Introdctor echniqes or 3-D Compter Vision, rcco and Verri Chapter 8: Motion Continos Motion Motion Consider a ideo camera moing continosl along a trajector (rotating & translating). How do points in the image moe? What does that tell s abot the 3-D motion & scene strctre? When objects moe at eqal speed, those more remote seem to moe more slowl. - Eclid, 300 BC Simplest Idea or ideo processing Image Dierences Gien image I(,,t) and I(,, t+t), compte I(,, t+t) - I(,,t). his is partial deriatie: I t I At object bondaries, is large and is a t ce or segmentation Does not indicate which wa objects are moing Backgrond Sbtraction Gather image I(,,t 0 ) o backgrond withot objects o interest (perhaps compted oer aerage oer man images). At time t, piels where I(,,t)-I(,,t 0 ) > are labeled as coming rom oregrond objects Raw Image Foregrond region 1

2 he Motion Field Where in the image did a point moe? he Motion Field Down and let What cases a motion ield? 1. Camera moes (translates, rotates). Objects in scene moe rigidl 3. Objects articlate (pliers, hmans, animals) 4. Objects bend and deorm (ish) 5. Blowing smoke, clods Is motion estimation inherent in hmans? Demo Rigid Motion and the Motion Field Rigid Motion: General Case p P Position and orientation o a rigid bod Rotation Matri & ranslation ector Rigid Motion: Velocit Vector: Anglar Velocit Vector: (or ) p p

3 3 General Motion p p Sbstitte where p=(,,) Motion Field Eqation : Components o 3-D linear motion Anglar elocit ector (,): Image point coordinates : depth : ocal length Pre ranslation Forward ranslation & Focs o Epansion [Gibson, 1950] Pre ranslation Radial abot FOE Parallel ( FOE point at ininit) = 0 Motion parallel to image plane Pre Rotation: =0 Independent o Independent o Onl nction o (,), and

4 Rotational MOION FIELD he instantaneos elocit o points in an image Pre translation and pre rotation: Motion Field on Sphere PURE ROAION = (0,0,1) Motion Field Eqation: Estimate Depth Optical Flow I, and are known or measred, then or each image point (,), one can sole or the depth gien measred motion (d/, d/) at (,). Optical Flow: Where do piels moe to? Estimating the motion ield rom images 1. Featre-based (Sect o rcco & Verri) 1. Detect (corner-like) eatres in an image. Search or the same eatres nearb (eatre tracking). Dierential techniqes (Sect ) 4

5 Problem Deinition: Optical Flow Optical Flow Motion Field How to estimate piel motion rom image H to image I? Find piel correspondences Gien a piel in H, look or nearb piels o the same color in I Ke assmptions color constanc: a point in H looks the same in image I For grascale images, this is brightness constanc small motion: points do not moe er ar Motion ield eists bt no optical low No motion ield bt shading changes Deinition o optical low OPICAL FLOW = apparent motion o brightness patterns Ideall, the optical low is the projection o the three-dimensional elocit ectors on the image (, ) time t Optical Flow Constraint Eqation (, ) time t t ( t, t) 1. Assme brightness o patch remains same in both images: Optical Flow: Velocities (, ) Displacement: I( t, t,t t) I(,,t) (, ) ( t, t). Assme small motion: (alor epansion o LHS p to irst order) I(,,t) I I I t I(,,t) t (, ) time t Optical Flow Constraint Eqation (, ) time t t ( t, t) Optical Flow: Velocities (, ) Displacement: 3. Sbtracting I(,,t) rom both sides and diiding b t I t I t I t 0 4. Assme small interal, this becomes: d I d I I (, ) ( t, t) Mathematical ormlation [Note change o notation: image coordinates now (,), not (,)] I (,,t) = brightness at image point (,) at time t Consider scene (or camera) to be moing, so (t), (t) Brightness constanc assmption: d d I( t, t, t t) I(,, t) Optical low constraint eqation : I I I 0 t t 0 di d d di 0 5

6 Soling or low Optical low constraint eqation : di I We can measre d We want to sole or I d I I I,, t d d, One eqation, two nknowns I t 0 Measrements I I I I I I t t Flow ector d d he component o the optical low in the direction o the image gradient. Optical Flow Constraint Apparentl an apertre problem What is the correspondence o P & P wo was to get low 1. hink globall, and reglarie oer image Contor plots o image intensit in two images. Look oer window and assme constant motion in the window 6

7 de(, ) d de(, ) d I I I I I I I I t t 0 0 Edge Low tetre region large gradients, all the same large 1, small gradients hae small magnitde small 1, small High tetred region Some ariants Iteratie reinement Coarse to ine (image pramids) Local/global motion models Robst estimation gradients are dierent, large magnitdes large 1, large 7

8 Reisiting the small motion assmption Is this motion small enogh? Probabl not it s mch larger than one piel ( nd order terms dominate) How might we sole this problem? Coarse-to-ine optical low estimation rn iteratie L-K warp & psample rn iteratie L-K... image J image I Gassian pramid o image J Gassian pramid o image I 8

9 Mlti-resoltion Lcas Kanade Algorithm Motion Model Eample: Aine Motion Net Lectre racking Reading: Chapter 11: racking 9