Lecture 13: Tracking mo3on features op3cal flow
|
|
|
- Jeffry Wilson
- 5 years ago
- Views:
Transcription
1 Lecture 13: Tracking mo3on features op3cal flow Professor Fei- Fei Li Stanford Vision Lab Lecture 13-1!
2 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons (Problem Set 3 (Q1)) Lecture 13-2!
3 From images to videos A video is a sequence of frames captured over 3me Now our image data is a func3on of space (x, y) and 3me (t) Lecture 13-3!
4 Mo3on es3ma3on techniques Op3cal flow Recover image mo3on at each pixel from spa3o- temporal image brightness varia3ons (op3cal flow) Feature- tracking Extract visual features (corners, textured areas) and track them over mul3ple frames Lecture 13-4!
5 Op3cal flow Vector field func3on of the spa3o- temporal image brightness varia3ons Picture courtesy of Selim Temizer - Learning and Intelligent Systems (LIS) Group, MIT Lecture 13-5!
6 Feature- tracking Courtesy of Jean- Yves Bouguet Vision Lab, California Ins3tute of Technology Lecture 13-6!
7 Feature- tracking Courtesy of Jean- Yves Bouguet Vision Lab, California Ins3tute of Technology Lecture 13-7!
8 Op3cal flow Defini3on: op3cal flow is the apparent mo3on of brightness paderns in the image Note: apparent mo3on can be caused by ligh3ng changes without any actual mo3on Think of a uniform rota3ng sphere under fixed ligh3ng vs. a sta3onary sphere under moving illumina3on GOAL: Recover image mo3on at each pixel from op3cal flow Lecture 13-8!
9 Es3ma3ng op3cal flow I(x,y,t 1) I(x,y,t) Given two subsequent frames, es3mate the apparent mo3on field u(x,y), v(x,y) between them Key assump3ons Brightness constancy: projec3on of the same point looks the same in every frame Small mo7on: points do not move very far Spa7al coherence: points move like their neighbors Lecture 13-9!
10 Lecture 13 -! t y x I y x v I y x u I t y x I t u y u x I ), ( ), ( 1),, ( ),, ( Brightness Constancy Equa3on: ), ( 1),, ( ),, ( ), ( t y x y x v y u x I t y x I + + = Linearizing the right side using Taylor expansion: The brightness constancy constraint I(x,y,t 1) I(x,y,t) t y x I v I u I Hence, Image deriva3ve along x [ ] 0 I v u I t T = + t y x I y x v I y x u I t y x I t u y u x I + + = + + ), ( ), ( 1),, ( ),, ( 10
11 The brightness constancy constraint Can we use this equa3on to recover image mo3on (u,v) at each pixel? I [ ] T u v + I = 0 How many equa3ons and unknowns per pixel? One equa3on (this is a scalar equa3on!), two unknowns (u,v) t The component of the flow perpendicular to the gradient (i.e., parallel to the edge) cannot be measured If (u, v ) sa3sfies the equa3on, so does (u+u, v+v ) if I [ u' v' ] T = 0 (u,v ) gradient edge (u,v) (u+u,v+v ) Lecture 13-11!
12 The aperture problem Actual mo7on Lecture 13-12!
13 The aperture problem Perceived mo7on Lecture 13-13!
14 The barber pole illusion hdp://en.wikipedia.org/wiki/barberpole_illusion Lecture 13-14!
15 The barber pole illusion hdp://en.wikipedia.org/wiki/barberpole_illusion Lecture 13-15!
16 Aperture problem cont d * From Marc Pollefeys COMP Lecture 13-16!
17 Solving the ambiguity B. Lucas and T. Kanade. An itera3ve image registra3on technique with an applica3on to stereo vision. In Proceedings of the Interna6onal Joint Conference on Ar6ficial Intelligence, pp , How to get more equa3ons for a pixel? Spa7al coherence constraint: Assume the pixel s neighbors have the same (u,v) If we use a 5x5 window, that gives us 25 equa3ons per pixel Lecture 13-17!
18 Lucas- Kanade flow Overconstrained linear system: Lecture 13-18!
19 Condi3ons for solvability When is this system solvable? What if the window contains just a single straight edge? Lecture 13-19!
20 Lucas- Kanade flow Overconstrained linear system Least squares solu3on for d given by The summa3ons are over all pixels in the K x K window Lecture 13-20!
21 Condi3ons for solvability Op3mal (u, v) sa3sfies Lucas- Kanade equa3on When is This Solvable? A T A should be inver3ble A T A should not be too small due to noise eigenvalues λ 1 and λ 2 of A T A should not be too small A T A should be well- condi3oned λ 1 / λ 2 should not be too large (λ 1 = larger eigenvalue) Does this remind anything to you? Lecture 13-21!
22 M = A T A is the second moment matrix! (Harris corner detector ) Eigenvectors and eigenvalues of A T A relate to edge direc3on and magnitude The eigenvector associated with the larger eigenvalue points in the direc3on of fastest intensity change The other eigenvector is orthogonal to it Lecture 13-22!
23 Interpre3ng the eigenvalues Classifica3on of image points using eigenvalues of the second moment matrix: λ 2 Edge λ 2 >> λ 1 Corner λ 1 and λ 2 are large, λ 1 ~ λ 2 λ 1 and λ 2 are small Flat region Edge λ 1 >> λ 2 λ 1 Lecture 13-23!
24 Edge gradients very large or very small large λ 1, small λ 2 Lecture 13-24!
25 Low- texture region gradients have small magnitude small λ 1, small λ 2 Lecture 13-25!
26 High- texture region gradients are different, large magnitudes large λ 1, large λ 2 Lecture 13-26!
27 What are good features to track? Can measure quality of features from just a single image Hence: tracking Harris corners (or equivalent) guarantees small error sensi3vity! à Implemented in Open CV Lecture 13-27!
28 Recap Key assump3ons (Errors in Lucas- Kanade) Small mo7on: points do not move very far Brightness constancy: projec3on of the same point looks the same in every frame Spa7al coherence: points move like their neighbors Lecture 13-28!
29 Revisi3ng the small mo3on assump3on Is this mo3on small enough? Probably not it s much larger than one pixel (2 nd order terms dominate) How might we solve this problem? * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 13-29!
30 Reduce the resolu3on! * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 13-30!
31 Mul3- resolu3on Lucas Kanade Algorithm Lecture 13-31!
32 Coarse- to- fine op3cal flow es3ma3on u=1.25 pixels u=2.5 pixels u=5 pixels image 1 H u=10 pixels image 2 I Gaussian pyramid of image 1 Gaussian pyramid of image 2 Lecture 13-32!
33 Itera3ve Refinement Iterative Lukas-Kanade Algorithm 1. Estimate velocity at each pixel by solving Lucas- Kanade equations 2. Warp I(t-1) towards I(t) using the estimated flow field - use image warping techniques 3. Repeat until convergence * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 13-33!
34 Coarse- to- fine op3cal flow es3ma3on run itera3ve L- K run itera3ve L- K warp & upsample... image 1 J Gaussian pyramid of image 1 (t) image 2 I Gaussian pyramid of image 2 (t+1) Lecture 13-34!
35 Op3cal Flow Results * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 13-35!
36 Op3cal Flow Results hdp:// OpenCV * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Lecture 13-36!
37 Recap Key assump3ons (Errors in Lucas- Kanade) Small mo7on: points do not move very far Brightness constancy: projec3on of the same point looks the same in every frame Spa7al coherence: points move like their neighbors Lecture 13-37!
38 Mo3on segmenta3on How do we represent the mo3on in this scene? Lecture 13-38!
39 Mo3on segmenta3on J. Wang and E. Adelson. Layered Representa3on for Mo3on Analysis. CVPR Break image sequence into layers each of which has a coherent (affine) mo3on Lecture 13-39!
40 What are layers? Each layer is defined by an alpha mask and an affine mo3on model J. Wang and E. Adelson. Layered Representa3on for Mo3on Analysis. CVPR Lecture 13-40!
41 Lecture 13 -! Subs3tu3ng into the brightness constancy equa3on: y a x a a y x v y a x a a y x u ), ( ), ( + + = + + = t y x I v I u I Affine mo3on 41
42 Lecture 13 -! 0 ) ( ) ( t y x I y a x a a I y a x a a I Subs3tu3ng into the brightness constancy equa3on: y a x a a y x v y a x a a y x u ), ( ), ( + + = + + = Each pixel provides 1 linear constraint in 6 unknowns [ ] = t y x I y a x a a I y a x a a I a Err ) ( ) ( ) ( Least squares minimiza3on: Affine mo3on 42
43 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene Lecture 13-43!
44 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene 2. Iterate un3l convergence: Assign each pixel to best hypothesis Pixels with high residual error remain unassigned Lecture 13-44!
45 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene Lecture 13-45!
46 How do we es3mate the layers? 1. Obtain a set of ini3al affine mo3on hypotheses Divide the image into blocks and es3mate affine mo3on parameters in each block by least squares Eliminate hypotheses with high residual error Map into mo3on parameter space Perform k- means clustering on affine mo3on parameters Merge clusters that are close and retain the largest clusters to obtain a smaller set of hypotheses to describe all the mo3ons in the scene 2. Iterate un3l convergence: Assign each pixel to best hypothesis Pixels with high residual error remain unassigned Perform region filtering to enforce spa3al constraints Re- es3mate affine mo3ons in each region Lecture 13-46!
47 Example result J. Wang and E. Adelson. Layered Representation for Motion Analysis. CVPR Lecture 13-47!
48 Tracking Sources: Kristen Grauman, Deva Ramanan Lecture 13-48!
49 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons Lecture 13-49!
50 Mo3on es3ma3on techniques Op3cal flow Recover image mo3on at each pixel from spa3o- temporal image brightness varia3ons (op3cal flow) Feature- tracking Extract visual features (corners, textured areas) and track them over mul3ple frames Shi- Tomasi feature tracker Tracking with dynamics Lecture 13-50!
51 Feature tracking So far, we have only considered op3cal flow es3ma3on in a pair of images If we have more than two images, we can compute the op3cal flow from each frame to the next Given a point in the first image, we can in principle reconstruct its path by simply following the arrows Lecture 13-51!
52 Tracking challenges Ambiguity of op3cal flow Find good features to track Large mo3ons Discrete search instead of Lucas- Kanade Changes in shape, orienta3on, color Allow some matching flexibility Occlusions, dis- occlusions Need mechanism for dele3ng, adding new features Driy errors may accumulate over 3me Need to know when to terminate a track Lecture 13-52!
53 Shi- Tomasi feature tracker J. Shi and C. Tomasi. Good Features to Track. CVPR Find good features using eigenvalues of second- moment matrix Key idea: good features to track are the ones that can be tracked reliably From frame to frame, track with Lucas- Kanade and a pure transla6on model More robust for small displacements, can be es3mated from smaller neighborhoods Check consistency of tracks by affine registra3on to the first observed instance of the feature Affine model is more accurate for larger displacements Comparing to the first frame helps to minimize driy Lecture 13-53!
54 Tracking example Lecture 13-54!
55 Tracking with dynamics Key idea: Given a model of expected mo3on, predict where objects will occur in next frame, even before seeing the image Restrict search for the object Improved es3mates since measurement noise is reduced by trajectory smoothness Lecture 13-55!
56 Tracking with dynamics initial position prediction measurement update y y y y x x x x The Kalman filter: Method for tracking linear dynamical models in Gaussian noise The predicted/corrected state distribu3ons are Gaussian Need to maintain the mean and covariance Calcula3ons are easy (all the integrals can be done in closed form) Lecture 13-56!
57 2D Target tracking using Kalman filter in MATLAB by AliReza KashaniPour hdp:// Lecture 13-57!
58 What we will learn today? Introduc3on Op3cal flow Feature tracking Applica3ons Lecture 13-58!
59 Uses of mo3on Tracking features Segmen3ng objects based on mo3on cues Learning dynamical models Improving video quality Mo3on stabiliza3on Super resolu3on Tracking objects Recognizing events and ac3vi3es Lecture 13-59!
60 Es3ma3ng 3D structure Lecture 13-60!
61 Segmen3ng objects based on mo3on cues Background subtrac3on A sta3c camera is observing a scene Goal: separate the sta3c background from the moving foreground Lecture 13-61!
62 Segmen3ng objects based on mo3on cues Mo3on segmenta3on Segment the video into mul3ple coherently moving objects S. J. Pundlik and S. T. Birchfield, Mo3on Segmenta3on at Any Speed, Proceedings of the Bri3sh Machine Vision Conference (BMVC) 2006 Lecture 13-62!
63 Tracking objects Z.Yin and R.Collins, "On- the- fly Object Modeling while Tracking," IEEE Computer Vision and PaJern Recogni6on (CVPR '07), Minneapolis, MN, June Lecture 13-63!
64 Synthesizing dynamic textures Lecture 13-64!
65 Super- resolu3on Example: A set of low quality images Lecture 13-65!
66 Super- resolu3on Each of these images looks like this: Lecture 13-66!
67 Super- resolu3on The recovery result: Lecture 13-67!
68 Recognizing events and ac3vi3es Tracker D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their Appearance. PAMI Lecture 13-68!
69 Recognizing events and ac3vi3es Juan Carlos Niebles, Hongcheng Wang and Li Fei- Fei, Unsupervised Learning of Human Ac7on Categories Using Spa7al- Temporal Words, (BMVC), Edinburgh, Lecture 13-69!
70 Recognizing events and ac3vi3es Crossing Talking Queuing Dancing jogging W. Choi & K. Shahid & S. Savarese WMC 2010 Lecture 13-70!
71 W. Choi, K. Shahid, S. Savarese, "What are they doing? : Collec3ve Ac3vity Classifica3on Using Spa3o- Temporal Rela3onship Among People", 9th Interna3onal Workshop on Visual Surveillance (VSWS09) in conjuc3on with ICCV 09 Lecture 13-71!
72 Op3cal flow without mo3on! Lecture 13-72!
73 What we have learned today? Introduc3on Op3cal flow Feature tracking Applica3ons (Problem Set 3 (Q1)) Lecture 13-73!
Lecture 14: Tracking mo3on features op3cal flow
Lecture 14: Tracking mo3on features op3cal flow Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei- Fei Li Stanford Vision Lab Lecture 14-1 What we will learn today? Introduc3on Op3cal flow Feature
Optical flow and tracking
EECS 442 Computer vision Optical flow and tracking Intro Optical flow and feature tracking Lucas-Kanade algorithm Motion segmentation Segments of this lectures are courtesy of Profs S. Lazebnik S. Seitz,
Feature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,
Feature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
Peripheral drift illusion
Peripheral drift illusion Does it work on other animals? Computer Vision Motion and Optical Flow Many slides adapted from J. Hays, S. Seitz, R. Szeliski, M. Pollefeys, K. Grauman and others Video A video
EECS 556 Image Processing W 09
EECS 556 Image Processing W 09 Motion estimation Global vs. Local Motion Block Motion Estimation Optical Flow Estimation (normal equation) Man slides of this lecture are courtes of prof Milanfar (UCSC)
CS 4495 Computer Vision Motion and Optic Flow
CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS4 is out, due Sunday Oct 27 th. All relevant lectures posted Details about Problem Set: You may *not* use built in Harris
Motion and Optical Flow. Slides from Ce Liu, Steve Seitz, Larry Zitnick, Ali Farhadi
Motion and Optical Flow Slides from Ce Liu, Steve Seitz, Larry Zitnick, Ali Farhadi We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
Ninio, J. and Stevens, K. A. (2000) Variations on the Hermann grid: an extinction illusion. Perception, 29,
Ninio, J. and Stevens, K. A. (2000) Variations on the Hermann grid: an extinction illusion. Perception, 29, 1209-1217. CS 4495 Computer Vision A. Bobick Sparse to Dense Correspodence Building Rome in
Visual motion. Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys
Visual motion Man slides adapted from S. Seitz, R. Szeliski, M. Pollefes Motion and perceptual organization Sometimes, motion is the onl cue Motion and perceptual organization Sometimes, motion is the
ECE Digital Image Processing and Introduction to Computer Vision
ECE592-064 Digital Image Processing and Introduction to Computer Vision Depart. of ECE, NC State University Instructor: Tianfu (Matt) Wu Spring 2017 Recap, SIFT Motion Tracking Change Detection Feature
Multi-stable Perception. Necker Cube
Multi-stable Perception Necker Cube Spinning dancer illusion, Nobuyuki Kayahara Multiple view geometry Stereo vision Epipolar geometry Lowe Hartley and Zisserman Depth map extraction Essential matrix
Computer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing
Computer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/76 Augmented Computing Many slides adapted from K. Grauman, S. Seitz, R. Szeliski, M. Pollefeys, S. Lazebnik Computer Vision Lecture 20 Motion and Optical Flow
Comparison between Motion Analysis and Stereo
MOTION ESTIMATION The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Octavia Camps (Northeastern); including their own slides. Comparison between Motion Analysis
(More) Algorithms for Cameras: Edge Detec8on Modeling Cameras/Objects. Connelly Barnes
(More) Algorithms for Cameras: Edge Detec8on Modeling Cameras/Objects Connelly Barnes Acknowledgment: Many slides from James Hays, also Derek Hoiem Grauman&Leibe 2008 Outline Edge Detec)on: Canny, etc.
Fundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F
Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental
Computer Vision Lecture 20
Computer Vision Lecture 2 Motion and Optical Flow Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de 28.1.216 Man slides adapted from K. Grauman, S. Seitz, R. Szeliski,
Lecture 16: Computer Vision
CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field
Lecture 16: Computer Vision
CS442/542b: Artificial ntelligence Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field Methods
CAP5415-Computer Vision Lecture 8-Mo8on Models, Feature Tracking, and Alignment. Ulas Bagci
CAP545-Computer Vision Lecture 8-Mo8on Models, Feature Tracking, and Alignment Ulas Bagci bagci@ucf.edu Readings Szeliski, R. Ch. 7 Bergen et al. ECCV 92, pp. 237-252. Shi, J. and Tomasi, C. CVPR 94, pp.593-6.
CS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 19: Optical flow http://en.wikipedia.org/wiki/barberpole_illusion Readings Szeliski, Chapter 8.4-8.5 Announcements Project 2b due Tuesday, Nov 2 Please sign
Finally: Motion and tracking. Motion 4/20/2011. CS 376 Lecture 24 Motion 1. Video. Uses of motion. Motion parallax. Motion field
Finally: Motion and tracking Tracking objects, video analysis, low level motion Motion Wed, April 20 Kristen Grauman UT-Austin Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, and S. Lazebnik
Matching. Compare region of image to region of image. Today, simplest kind of matching. Intensities similar.
Matching Compare region of image to region of image. We talked about this for stereo. Important for motion. Epipolar constraint unknown. But motion small. Recognition Find object in image. Recognize object.
Lucas-Kanade Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.
Lucas-Kanade Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
VC 11/12 T11 Optical Flow
VC 11/12 T11 Optical Flow Mestrado em Ciência de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informáticos Miguel Tavares Coimbra Outline Optical Flow Constraint Equation Aperture
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 11 140311 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Motion Analysis Motivation Differential Motion Optical
Overview. Video. Overview 4/7/2008. Optical flow. Why estimate motion? Motion estimation: Optical flow. Motion Magnification Colorization.
Overview Video Optical flow Motion Magnification Colorization Lecture 9 Optical flow Motion Magnification Colorization Overview Optical flow Combination of slides from Rick Szeliski, Steve Seitz, Alyosha
Optical flow. Cordelia Schmid
Optical flow Cordelia Schmid Motion field The motion field is the projection of the 3D scene motion into the image Optical flow Definition: optical flow is the apparent motion of brightness patterns in
Dense Image-based Motion Estimation Algorithms & Optical Flow
Dense mage-based Motion Estimation Algorithms & Optical Flow Video A video is a sequence of frames captured at different times The video data is a function of v time (t) v space (x,y) ntroduction to motion
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Today Go over Midterm Go over Project #3
COMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE
COMPUTER VISION 2017-2018 > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE OUTLINE Optical flow Lucas-Kanade Horn-Schunck Applications of optical flow Optical flow tracking Histograms of oriented flow Assignment
Computer Vision Lecture 18
Course Outline Computer Vision Lecture 8 Motion and Optical Flow.0.009 Bastian Leibe RWTH Aachen http://www.umic.rwth-aachen.de/multimedia leibe@umic.rwth-aachen.de Man slides adapted from K. Grauman,
CPSC 425: Computer Vision
1 / 49 CPSC 425: Computer Vision Instructor: Fred Tung ftung@cs.ubc.ca Department of Computer Science University of British Columbia Lecture Notes 2015/2016 Term 2 2 / 49 Menu March 10, 2016 Topics: Motion
Optical Flow-Based Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.
Optical Flow-Based Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object
Lecture 19: Motion. Effect of window size 11/20/2007. Sources of error in correspondences. Review Problem set 3. Tuesday, Nov 20
Lecture 19: Motion Review Problem set 3 Dense stereo matching Sparse stereo matching Indexing scenes Tuesda, Nov 0 Effect of window size W = 3 W = 0 Want window large enough to have sufficient intensit
CS664 Lecture #18: Motion
CS664 Lecture #18: Motion Announcements Most paper choices were fine Please be sure to email me for approval, if you haven t already This is intended to help you, especially with the final project Use
Representing Moving Images with Layers. J. Y. Wang and E. H. Adelson MIT Media Lab
Representing Moving Images with Layers J. Y. Wang and E. H. Adelson MIT Media Lab Goal Represent moving images with sets of overlapping layers Layers are ordered in depth and occlude each other Velocity
Autonomous Navigation for Flying Robots
Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 7.1: 2D Motion Estimation in Images Jürgen Sturm Technische Universität München 3D to 2D Perspective Projections
Motion and Tracking. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Motion and Tracking Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Motion Segmentation Segment the video into multiple coherently moving objects Motion and Perceptual Organization
Computer Vision II Lecture 4
Computer Vision II Lecture 4 Color based Tracking 29.04.2014 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Single-Object Tracking Background modeling
CSCI 1290: Comp Photo
CSCI 1290: Comp Photo Fall 2018 @ Brown University James Tompkin Many slides thanks to James Hays old CS 129 course, along with all its acknowledgements. CS 4495 Computer Vision A. Bobick Motion and Optic
ECE 634: Digital Video Systems Mo7on models: 1/19/17
ECE 634: Digital Video Systems Mo7on models: 1/19/17 Professor Amy Reibman MSEE 356 reibman@purdue.edu hip://engineering.purdue.edu/~reibman/ece634/index.html Outline Today: Mo7on models for mo7on es7ma7on
Visual Tracking. Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania.
Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania 1 What is visual tracking? estimation of the target location over time 2 applications Six main areas:
Particle Tracking. For Bulk Material Handling Systems Using DEM Models. By: Jordan Pease
Particle Tracking For Bulk Material Handling Systems Using DEM Models By: Jordan Pease Introduction Motivation for project Particle Tracking Application to DEM models Experimental Results Future Work References
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition. Motion Tracking. CS4243 Motion Tracking 1
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition Motion Tracking CS4243 Motion Tracking 1 Changes are everywhere! CS4243 Motion Tracking 2 Illumination change CS4243 Motion Tracking 3 Shape
CS231A Section 6: Problem Set 3
CS231A Section 6: Problem Set 3 Kevin Wong Review 6 -! 1 11/09/2012 Announcements PS3 Due 2:15pm Tuesday, Nov 13 Extra Office Hours: Friday 6 8pm Huang Common Area, Basement Level. Review 6 -! 2 Topics
Fi#ng & Matching Region Representa3on Image Alignment, Op3cal Flow
Fi#ng & Matching Region Representa3on Image Alignment, Op3cal Flow Lectures 5 & 6 Prof. Fergus Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros. Facebook 360 photos Panoramas How do we build
Optical Flow CS 637. Fuxin Li. With materials from Kristen Grauman, Richard Szeliski, S. Narasimhan, Deqing Sun
Optical Flow CS 637 Fuxin Li With materials from Kristen Grauman, Richard Szeliski, S. Narasimhan, Deqing Sun Motion and perceptual organization Sometimes, motion is the only cue Motion and perceptual
Optical flow. Cordelia Schmid
Optical flow Cordelia Schmid Motion field The motion field is the projection of the 3D scene motion into the image Optical flow Definition: optical flow is the apparent motion of brightness patterns in
3D Photography. Marc Pollefeys, Torsten Sattler. Spring 2015
3D Photography Marc Pollefeys, Torsten Sattler Spring 2015 Schedule (tentative) Feb 16 Feb 23 Mar 2 Mar 9 Mar 16 Mar 23 Mar 30 Apr 6 Apr 13 Apr 20 Apr 27 May 4 May 11 May 18 May 25 Introduction Geometry,
Image processing and features
Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry
Displacement estimation
Displacement estimation Displacement estimation by block matching" l Search strategies" l Subpixel estimation" Gradient-based displacement estimation ( optical flow )" l Lukas-Kanade" l Multi-scale coarse-to-fine"
Capturing, Modeling, Rendering 3D Structures
Computer Vision Approach Capturing, Modeling, Rendering 3D Structures Calculate pixel correspondences and extract geometry Not robust Difficult to acquire illumination effects, e.g. specular highlights
Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection
CS201: Computer Vision Introduction to Tracking
CS201: Computer Vision Introduction to Tracking John Magee 18 November 2014 Slides courtesy of: Diane H. Theriault Question of the Day How can we represent and use motion in images? 1 What is Motion? Change
Visual Tracking. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania
Visual Tracking Antonino Furnari Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania furnari@dmi.unict.it 11 giugno 2015 What is visual tracking? estimation
Motion Estimation. There are three main types (or applications) of motion estimation:
Members: D91922016 朱威達 R93922010 林聖凱 R93922044 謝俊瑋 Motion Estimation There are three main types (or applications) of motion estimation: Parametric motion (image alignment) The main idea of parametric motion
Motion. 1 Introduction. 2 Optical Flow. Sohaib A Khan. 2.1 Brightness Constancy Equation
Motion Sohaib A Khan 1 Introduction So far, we have dealing with single images of a static scene taken by a fixed camera. Here we will deal with sequence of images taken at different time intervals. Motion
Face Detec<on & Tracking using KLT Algorithm
Laboratory of Image Processing Face Detec
Wikipedia - Mysid
Wikipedia - Mysid Erik Brynjolfsson, MIT Filtering Edges Corners Feature points Also called interest points, key points, etc. Often described as local features. Szeliski 4.1 Slides from Rick Szeliski,
3D Vision. Viktor Larsson. Spring 2019
3D Vision Viktor Larsson Spring 2019 Schedule Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar 25 Apr 1 Apr 8 Apr 15 Apr 22 Apr 29 May 6 May 13 May 20 May 27 Introduction Geometry, Camera Model, Calibration Features,
Lecture 20: Tracking. Tuesday, Nov 27
Lecture 20: Tracking Tuesday, Nov 27 Paper reviews Thorough summary in your own words Main contribution Strengths? Weaknesses? How convincing are the experiments? Suggestions to improve them? Extensions?
Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects
Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
Corner Detection. Harvey Rhody Chester F. Carlson Center for Imaging Science Rochester Institute of Technology
Corner Detection Harvey Rhody Chester F. Carlson Center for Imaging Science Rochester Institute of Technology rhody@cis.rit.edu April 11, 2006 Abstract Corners and edges are two of the most important geometrical
Computer Vision II Lecture 4
Course Outline Computer Vision II Lecture 4 Single-Object Tracking Background modeling Template based tracking Color based Tracking Color based tracking Contour based tracking Tracking by online classification
Spatial track: motion modeling
Spatial track: motion modeling Virginio Cantoni Computer Vision and Multimedia Lab Università di Pavia Via A. Ferrata 1, 27100 Pavia virginio.cantoni@unipv.it http://vision.unipv.it/va 1 Comparison between
Horn-Schunck and Lucas Kanade 1
Horn-Schunck and Lucas Kanade 1 Lecture 8 See Sections 4.2 and 4.3 in Reinhard Klette: Concise Computer Vision Springer-Verlag, London, 2014 1 See last slide for copyright information. 1 / 40 Where We
Face Tracking : An implementation of the Kanade-Lucas-Tomasi Tracking algorithm
Face Tracking : An implementation of the Kanade-Lucas-Tomasi Tracking algorithm Dirk W. Wagener, Ben Herbst Department of Applied Mathematics, University of Stellenbosch, Private Bag X1, Matieland 762,
Local Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
Optic Flow and Basics Towards Horn-Schunck 1
Optic Flow and Basics Towards Horn-Schunck 1 Lecture 7 See Section 4.1 and Beginning of 4.2 in Reinhard Klette: Concise Computer Vision Springer-Verlag, London, 2014 1 See last slide for copyright information.
BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy
BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving
SURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD
SURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD M.E-II, Department of Computer Engineering, PICT, Pune ABSTRACT: Optical flow as an image processing technique finds its applications
Features Points. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Features Points Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Finding Corners Edge detectors perform poorly at corners. Corners provide repeatable points for matching, so
Edge and corner detection
Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements
CS395T Visual Recogni5on and Search. Gautam S. Muralidhar
CS395T Visual Recogni5on and Search Gautam S. Muralidhar Today s Theme Unsupervised discovery of images Main mo5va5on behind unsupervised discovery is that supervision is expensive Common tasks include
Motion. CS 554 Computer Vision Pinar Duygulu Bilkent University
1 Motion CS 554 Computer Vision Pinar Duygulu Bilkent University 2 Motion A lot of information can be extracted from time varying sequences of images, often more easily than from static images. e.g, camouflaged
Computer Vision I - Basics of Image Processing Part 2
Computer Vision I - Basics of Image Processing Part 2 Carsten Rother 07/11/2014 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
Towards the completion of assignment 1
Towards the completion of assignment 1 What to do for calibration What to do for point matching What to do for tracking What to do for GUI COMPSCI 773 Feature Point Detection Why study feature point detection?
Video Alignment. Literature Survey. Spring 2005 Prof. Brian Evans Multidimensional Digital Signal Processing Project The University of Texas at Austin
Literature Survey Spring 2005 Prof. Brian Evans Multidimensional Digital Signal Processing Project The University of Texas at Austin Omer Shakil Abstract This literature survey compares various methods
Marcel Worring Intelligent Sensory Information Systems
Marcel Worring worring@science.uva.nl Intelligent Sensory Information Systems University of Amsterdam Information and Communication Technology archives of documentaries, film, or training material, video
Computer Vision I - Filtering and Feature detection
Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
Lecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
Visual Tracking (1) Feature Point Tracking and Block Matching
Intelligent Control Systems Visual Tracking (1) Feature Point Tracking and Block Matching Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
Final Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
Kanade Lucas Tomasi Tracking (KLT tracker)
Kanade Lucas Tomasi Tracking (KLT tracker) Tomáš Svoboda, svoboda@cmp.felk.cvut.cz Czech Technical University in Prague, Center for Machine Perception http://cmp.felk.cvut.cz Last update: November 26,
Computer Vision I. Announcements. Fourier Tansform. Efficient Implementation. Edge and Corner Detection. CSE252A Lecture 13.
Announcements Edge and Corner Detection HW3 assigned CSE252A Lecture 13 Efficient Implementation Both, the Box filter and the Gaussian filter are separable: First convolve each row of input image I with
Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
Comparison Between The Optical Flow Computational Techniques
Comparison Between The Optical Flow Computational Techniques Sri Devi Thota #1, Kanaka Sunanda Vemulapalli* 2, Kartheek Chintalapati* 3, Phanindra Sai Srinivas Gudipudi* 4 # Associate Professor, Dept.
SIFT (Scale Invariant Feature Transform) descriptor
Local descriptors SIFT (Scale Invariant Feature Transform) descriptor SIFT keypoints at loca;on xy and scale σ have been obtained according to a procedure that guarantees illumina;on and scale invance.
Optic Flow and Motion Detection
Optic Flow and Motion Detection Computer Vision and Imaging Martin Jagersand Readings: Szeliski Ch 8 Ma, Kosecka, Sastry Ch 4 Image motion Somehow quantify the frame-to to-frame differences in image sequences.
Segmentation and Tracking of Partial Planar Templates
Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract
EE 264: Image Processing and Reconstruction. Image Motion Estimation I. EE 264: Image Processing and Reconstruction. Outline
1 Image Motion Estimation I 2 Outline 1. Introduction to Motion 2. Why Estimate Motion? 3. Global vs. Local Motion 4. Block Motion Estimation 5. Optical Flow Estimation Basics 6. Optical Flow Estimation
Spatial track: motion modeling
Spatial track: motion modeling Virginio Cantoni Computer Vision and Multimedia Lab Università di Pavia Via A. Ferrata 1, 27100 Pavia virginio.cantoni@unipv.it http://vision.unipv.it/va 1 Comparison between
CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
Announcements. Computer Vision I. Motion Field Equation. Revisiting the small motion assumption. Visual Tracking. CSE252A Lecture 19.
Visual Tracking CSE252A Lecture 19 Hw 4 assigned Announcements No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Motion Field Equation Measurements I x = I x, T: Components
Local features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
Computer Vision for HCI. Motion. Motion
Computer Vision for HCI Motion Motion Changing scene may be observed in a sequence of images Changing pixels in image sequence provide important features for object detection and activity recognition 2
CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix