Motion. 1 Introduction. 2 Optical Flow. Sohaib A Khan. 2.1 Brightness Constancy Equation
|
|
- Susanna Adams
- 5 years ago
- Views:
Transcription
1 Motion Sohaib A Khan 1 Introduction So far, we have dealing with single images of a static scene taken by a fixed camera. Here we will deal with sequence of images taken at different time intervals. Motion of an object in 3D or the camera induces 2D motion of pixels on the image plane. This motion is called optical flow. We will first present a method to compute local optical flow at each pixel, followed by a method to compute global flow, which fits the affine model. 2 Optical Flow Optical flow may be defined as a flow-field u(x, y),v(x, y) where u(x, y) is the velocity of pixel (x, y) in the x-direction, and v(x, y) in the y-direction. Example optical flow fields are shown in Figure 1. It can be realized from this figure that optical flow is a powerful feature for object segmentation. In addition, there are several applications that stem from optical flow computation, like object-based compression (MPEG-4), image stabilization and gesture recognition. 2.1 Brightness Constancy Equation Let a 3D function, f(x, y, t), where x and y are the spatial coordinates, and t is the time coordinate, denote the image sequence. Then, f(x 1,y 1,t 1 )isthegraylevelatpixel(x 1,y 1 )attimet 1. We assume that if there is a small change dx, dy and dt in x, y and t, there is no change in the gray-levels, that is: f(x, y, t) =f(x + dx, y + dy, t + dt) (1) This equation represents the brightness constancy assumption. That is, over small time intervals, pixels can experience small displacements but with no change in color. This assumption is not always true in real world sequences, because of non-lambertian object surfaces, changes in distance from light sources, and camera noise. However it is a reasonable simplifying assumption, based on which we can easily derive equations for computing u and v. Recall that the Taylor series expansion of a function f(x) aboutapointx = a is given by: f(x) =f(a)+(x a)f (a)+ 1 2! (x a)2 f (a)+... (2) Thus, the Taylor series expansion of right hand side of Equation 1 around x, y, t, is: f(x + dx, y + dy, t + dt) =f(x, y, t)+dx f f + dy x y + dt f +... (3) t Adapted from Fundamentals of Computer Vision by Mubarak Shah c 1992, and other sources 1
2 Figure 1: Examples of optical flow fields: (a) translation, (b) rotation, (c) zoom, (d) unzoom are flow fields that will fit a global model like affine or projective. (e) shows an image from a sequence whose optical flow is shown in (f). Here different image regions are moving at different velocities that do not fit a global 2D displacement model. Ignoring higher order terms and substituting in Equation 1, we get: f(x, y, t) =f(x, y, t)+dx f f + dy x y + dt f t. (4) The above equation can be simplified as: f x dx + f y dy + f t dt =0, (5) where f x = f x, f y = f y, f t = f t are the x-, y- andt-derivatives respectively. These derivatives can be computed by convolving the sequence f(x, y, t) with the masks shown in Figure 2. Dividing each term in the above equation by dt, weget: f x u + f y v + ft =0, (6) where u = dx dt and v = dy dt is the optical flow. This equation is called the brightness constancy equation. It consists of two unknowns, u and v, and therefore a unique solution for these two unknowns does not exist, based on a information available at a single pixel. In fact, the equation presents a linear constraint on the possible solutions of u and v, which can be seen if it is rewritten as: v = f x u f t. (7) f y f y This is the equation of a straight line in uv-space. There are several possible solutions of u, v, which lie along this line, as shown in Figure 3. Let (û, ˆv) be the correct solution. This vector can 2
3 Figure 2: Derivative Masks: The axis convention (left) and the derivative masks that conform to this convention. Note that in this convention, optical flow vectors go from I t to I t+1. be divided into two components, one along the straight line denoted by p and the other along the perpendicular line denoted by d. We can show that since cos α = d/ ft f x and cos(90 α) =d/ ft f y, f therefore d = (f t 2 x +fy 2 ). Therefore, knowing the derivatives f x, f y and f t, we can only compute the normal component, d, of optical flow. However, the parallel component p cannot be computed directly from the derivatives Example It is instructive to do a simple example to clearly understand the concept of normal and parallel components of optical flow. Consider a foreground object which has translated by û = 1, ˆv = 1 between two frames 4. We are interested in finding optical flow at the point marked x. Applying the masks in Figure 2 at x (assuming origin of the mask to be bottom right corner), we get f x (x) =0,f y (x) =2andf t (x) = 2. This means that the possible solutions of (u, v) lie along the line v = 1 in the uv-space. This makes sense, because if we look at a localized neighborhood around point x, we can only determine that the edge has moved by one pixel in the horizontal direction. All variations in the vertical direction, for example (u =0,v =1),(u = 1,v =1),(u =3,v =1) will generate exactly the same local variations in the image, and hence the same derivatives. This example illustrates an important problem with the brightness constancy equation, stated as the aperture problem. This problem is illustrated in Figure 5 and can be stated as follows: The component of the motion field in the direction orthogonal to the spatial image gradient is not constrained by the brightness constancy equation. 3 Lucas-Kanade Method Lucas-Kanade s method of finding optical flow relies on the least-squares solution of (u, v) over a small neighborhood. The idea is very simple and is as follows. We know that a single point yields one equation from which two unknowns cannot be recovered. However, if we assume that brightness constancy assumption holds for a small neighborhood around the point (typically 3x3 3
4 Figure 3: Optical flow constraint line in uv-space. d is the length of the perpendicular from the origin to the line, α is the angle the perpendicular makes with the x-axis. (û, ˆv) is one possible solution, which can be divided into two components: p long the constraint line, and d which is perpendicular to the constraint line. Figure 4: Example of an image sequence with (1, 1) translation at all points between two frames. Derivative masks are applied at x (affecting pixels enclosed in blue square). If white pixels are 1 and black are 0, then f x =0,f y = 2, f t = 2. The possible solutions of (u, v) lie along the line v = 1 in the uv-space (right). 4
5 Figure 5: The aperture problem: the black and gray lines represent two positions of the same image line in two consecutive frames. The image velocity, perceived in the image on the left through a small aperture is only the normal component d. The actual image velocity is shown on right, as u. or 5x5 neighborhood), then each point will yield one equation, but this set of equations will still have only two unknowns. This yields an over-constrained linear system, which can be solved by least-squares method. Formally, consider a 3x3 neighborhood for which brightness constancy assumption holds, i.e. we assume that the entire neighborhood has moved over interval dt with velocity (u, v). Then: f x1 u + f y1 v = f t1 (8) f x2 u + f y2 v = f t2. f x9 u + f y9 v = f t9 If we define A = f x1 f x2. f x9 f y1 f y2. f y9 then this gives us a linear system of the form B = f t1 f t2. f t9 u = [ u v ], (9) which can be solved by taking the pseudo-inverse: Au = B, (10) A T Au = A T B (11) ( 1 u = A A) T A T B (12) This solution is the least-squares solution, which means that it finds the values of (u, v) such that the square of the error is the minimum. This can be realized by deriving Eq. 11 in an alternate 5
6 manner. Consider the error term e = i (f x u + f y v + f t ) 2. (13) This error should be ideally zero over all points in the neighborhood, so we can find optimal values of (u, v) by minimizing this equation. e u =2 (f xi u + f yi v + f ti )(f xi ) = 0 (14) i e v =2 (f xi u + f yi v + f ti )(f yi )=0 i We can simplify these two equations and write them as: [ i f xi 2 i f ][ ] [ xif yi u i i f xif yi i f yi 2 = f ] xif ti v i f, (15) yif ti which is simply an expanded form of Eq Lucas-Kanade with Pyramids Typically the derivative masks are small in size and therefore cannot capture faster moving objects in the scene. Therefore there is a need to either use larger derivative masks, or to use smaller images. One technique to do this is to use pyramids. At the highest level of the pyramid, standard Lucas-Kanade is applied. The resulting flow vectors from each level are propagated to the next level, using interpolation for the intermediate values. They are then multiplied by two to compensate for the increased resolution at this level. The correction in the flow vectors is then computed by applying LK, but with the additional step that f t is computed after compensating for the known estimate of flow. Finally the correction is added to the initial estimate, to obtain optical flow at the current level. The algorithm is illustrated in Figure 6. 4 Global Affine Flow So far we have looked at the issue of computing optical flow at every pixel. Often, the sequence of images is such that the entire image is being deformed in a consistent manner. Such images are generated mostly because of camera motion. For example, if the camera is translating or zooming, then optical flow of an image in this sequence has global consistency. Assuming that the deformation between frames I and I is given by the affine transformation: x = a 1x + a 2 y + b 1 y = a 3 x + a 4y + b 2, (16) the optical flow at every pixel is also related to the pixel coordinates. x x = u = a 1 x + a 2 y + b 1 y y = v = a 3 x + a 4 y + b 2 (17) where a 1 = a 1 1anda 4 = a 4 1. This equation gives a global model for optical flow, i.e. (u, v) values over the entire image are related by this model. Given two images, the model parameters 6
7 Figure 6: Lucas-Kanade with pyramids algorithm a =(a 1,a 2,b 1,a 3,a 4,b 2 ) T can be recovered finding the value of a which minimize the error given by the brightness constancy equation. We define the error term over the image as: e = pixels (f t + f x u) 2, (18) where f x =[f x,f y ] T and u =[u, v] T. Note that Eq. 17 can be rewritten in terms of the unknowns as follows: a 1 [ ] [ ] a 2 u x y b 1 = v x y 1 a 3 a 4 b 2 u = Xa (19) Substituting u in Eq. 18, we get e = pixels (f t + f x Xa) 2. (20) This equation represents the combined deviation of the whole image from the brightness constancy equation when affine deformation of a is assumed. The optimal value of a is the one that minimizes e, which can be obtained by solving e a = 0. This gives us the following equation: ( ) X T f x fx T X u = X T f x f t (21) pixels 7 pixels
8 Figure 7: Iterative method for computing global flow. W denotes warping module, M denotes global motion estimation module (Eq. 21) and + denotes the process of combining two transformations. In practice, several iterations may be done at each level, which are not shown here for the sake of simplicity. The term ( ) pixels X T f x fx T X is a 6x6 matrix, which can be inverted to solve this linear system for the unknown parameters, a. Practically this process is also done using pyramids. The 2x2 derivative masks cannot capture large motion, and therefore the process is done iteratively at multiple resolutions. We are given two images I 1 and I 2. Ateachlevell of the pyramid, the transformation from the previous level a l 1 is used as the initial estimate. Image I 1 at this level is warped using this transformation 1. The remaining transformation δa l between the warped image I1 and I 2 is recovered using Eq. 21. The final transformation at this level is then the product of the homogeneous transformation matrices of a l 1 and δa l. This process is illustrated in Figure 7. The initial estimate at the highest level is taken to be the identity transformation. 1 Note that for warping, 1 will have to be added to a 1 and a 4 8
COMPUTER VISION > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE
COMPUTER VISION 2017-2018 > OPTICAL FLOW UTRECHT UNIVERSITY RONALD POPPE OUTLINE Optical flow Lucas-Kanade Horn-Schunck Applications of optical flow Optical flow tracking Histograms of oriented flow Assignment
More informationDense Image-based Motion Estimation Algorithms & Optical Flow
Dense mage-based Motion Estimation Algorithms & Optical Flow Video A video is a sequence of frames captured at different times The video data is a function of v time (t) v space (x,y) ntroduction to motion
More informationCS-465 Computer Vision
CS-465 Computer Vision Nazar Khan PUCIT 9. Optic Flow Optic Flow Nazar Khan Computer Vision 2 / 25 Optic Flow Nazar Khan Computer Vision 3 / 25 Optic Flow Where does pixel (x, y) in frame z move to in
More informationEECS 556 Image Processing W 09
EECS 556 Image Processing W 09 Motion estimation Global vs. Local Motion Block Motion Estimation Optical Flow Estimation (normal equation) Man slides of this lecture are courtes of prof Milanfar (UCSC)
More informationLeow Wee Kheng CS4243 Computer Vision and Pattern Recognition. Motion Tracking. CS4243 Motion Tracking 1
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition Motion Tracking CS4243 Motion Tracking 1 Changes are everywhere! CS4243 Motion Tracking 2 Illumination change CS4243 Motion Tracking 3 Shape
More informationMatching. Compare region of image to region of image. Today, simplest kind of matching. Intensities similar.
Matching Compare region of image to region of image. We talked about this for stereo. Important for motion. Epipolar constraint unknown. But motion small. Recognition Find object in image. Recognize object.
More informationRuch (Motion) Rozpoznawanie Obrazów Krzysztof Krawiec Instytut Informatyki, Politechnika Poznańska. Krzysztof Krawiec IDSS
Ruch (Motion) Rozpoznawanie Obrazów Krzysztof Krawiec Instytut Informatyki, Politechnika Poznańska 1 Krzysztof Krawiec IDSS 2 The importance of visual motion Adds entirely new (temporal) dimension to visual
More informationFinally: Motion and tracking. Motion 4/20/2011. CS 376 Lecture 24 Motion 1. Video. Uses of motion. Motion parallax. Motion field
Finally: Motion and tracking Tracking objects, video analysis, low level motion Motion Wed, April 20 Kristen Grauman UT-Austin Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, and S. Lazebnik
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationLecture 16: Computer Vision
CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field
More informationCS 4495 Computer Vision Motion and Optic Flow
CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS4 is out, due Sunday Oct 27 th. All relevant lectures posted Details about Problem Set: You may *not* use built in Harris
More informationCS 565 Computer Vision. Nazar Khan PUCIT Lectures 15 and 16: Optic Flow
CS 565 Computer Vision Nazar Khan PUCIT Lectures 15 and 16: Optic Flow Introduction Basic Problem given: image sequence f(x, y, z), where (x, y) specifies the location and z denotes time wanted: displacement
More informationLecture 16: Computer Vision
CS442/542b: Artificial ntelligence Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field Methods
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationMidterm Exam Solutions
Midterm Exam Solutions Computer Vision (J. Košecká) October 27, 2009 HONOR SYSTEM: This examination is strictly individual. You are not allowed to talk, discuss, exchange solutions, etc., with other fellow
More informationVC 11/12 T11 Optical Flow
VC 11/12 T11 Optical Flow Mestrado em Ciência de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informáticos Miguel Tavares Coimbra Outline Optical Flow Constraint Equation Aperture
More informationOptical flow and tracking
EECS 442 Computer vision Optical flow and tracking Intro Optical flow and feature tracking Lucas-Kanade algorithm Motion segmentation Segments of this lectures are courtesy of Profs S. Lazebnik S. Seitz,
More informationCS664 Lecture #18: Motion
CS664 Lecture #18: Motion Announcements Most paper choices were fine Please be sure to email me for approval, if you haven t already This is intended to help you, especially with the final project Use
More informationMotion and Optical Flow. Slides from Ce Liu, Steve Seitz, Larry Zitnick, Ali Farhadi
Motion and Optical Flow Slides from Ce Liu, Steve Seitz, Larry Zitnick, Ali Farhadi We live in a moving world Perceiving, understanding and predicting motion is an important part of our daily lives Motion
More informationComputer Vision for HCI. Motion. Motion
Computer Vision for HCI Motion Motion Changing scene may be observed in a sequence of images Changing pixels in image sequence provide important features for object detection and activity recognition 2
More informationOptic Flow and Basics Towards Horn-Schunck 1
Optic Flow and Basics Towards Horn-Schunck 1 Lecture 7 See Section 4.1 and Beginning of 4.2 in Reinhard Klette: Concise Computer Vision Springer-Verlag, London, 2014 1 See last slide for copyright information.
More informationOptical Flow Estimation
Optical Flow Estimation Goal: Introduction to image motion and 2D optical flow estimation. Motivation: Motion is a rich source of information about the world: segmentation surface structure from parallax
More informationPeripheral drift illusion
Peripheral drift illusion Does it work on other animals? Computer Vision Motion and Optical Flow Many slides adapted from J. Hays, S. Seitz, R. Szeliski, M. Pollefeys, K. Grauman and others Video A video
More informationNotes 9: Optical Flow
Course 049064: Variational Methods in Image Processing Notes 9: Optical Flow Guy Gilboa 1 Basic Model 1.1 Background Optical flow is a fundamental problem in computer vision. The general goal is to find
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 19: Optical flow http://en.wikipedia.org/wiki/barberpole_illusion Readings Szeliski, Chapter 8.4-8.5 Announcements Project 2b due Tuesday, Nov 2 Please sign
More informationSURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD
SURVEY OF LOCAL AND GLOBAL OPTICAL FLOW WITH COARSE TO FINE METHOD M.E-II, Department of Computer Engineering, PICT, Pune ABSTRACT: Optical flow as an image processing technique finds its applications
More informationVisual motion. Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys
Visual motion Man slides adapted from S. Seitz, R. Szeliski, M. Pollefes Motion and perceptual organization Sometimes, motion is the onl cue Motion and perceptual organization Sometimes, motion is the
More informationMotion and Tracking. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Motion and Tracking Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Motion Segmentation Segment the video into multiple coherently moving objects Motion and Perceptual Organization
More informationComparison between Motion Analysis and Stereo
MOTION ESTIMATION The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Octavia Camps (Northeastern); including their own slides. Comparison between Motion Analysis
More information16720 Computer Vision: Homework 3 Template Tracking and Layered Motion.
16720 Computer Vision: Homework 3 Template Tracking and Layered Motion. Instructor: Martial Hebert TAs: Varun Ramakrishna and Tomas Simon Due Date: October 24 th, 2011. 1 Instructions You should submit
More informationVisual Tracking (1) Feature Point Tracking and Block Matching
Intelligent Control Systems Visual Tracking (1) Feature Point Tracking and Block Matching Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
More informationLucas-Kanade Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.
Lucas-Kanade Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object
More informationLecture 19: Motion. Effect of window size 11/20/2007. Sources of error in correspondences. Review Problem set 3. Tuesday, Nov 20
Lecture 19: Motion Review Problem set 3 Dense stereo matching Sparse stereo matching Indexing scenes Tuesda, Nov 0 Effect of window size W = 3 W = 0 Want window large enough to have sufficient intensit
More informationMotion Analysis. Motion analysis. Now we will talk about. Differential Motion Analysis. Motion analysis. Difference Pictures
Now we will talk about Motion Analysis Motion analysis Motion analysis is dealing with three main groups of motionrelated problems: Motion detection Moving object detection and location. Derivation of
More informationLecture 20: Tracking. Tuesday, Nov 27
Lecture 20: Tracking Tuesday, Nov 27 Paper reviews Thorough summary in your own words Main contribution Strengths? Weaknesses? How convincing are the experiments? Suggestions to improve them? Extensions?
More informationEE 264: Image Processing and Reconstruction. Image Motion Estimation I. EE 264: Image Processing and Reconstruction. Outline
1 Image Motion Estimation I 2 Outline 1. Introduction to Motion 2. Why Estimate Motion? 3. Global vs. Local Motion 4. Block Motion Estimation 5. Optical Flow Estimation Basics 6. Optical Flow Estimation
More informationComputer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/76 Augmented Computing Many slides adapted from K. Grauman, S. Seitz, R. Szeliski, M. Pollefeys, S. Lazebnik Computer Vision Lecture 20 Motion and Optical Flow
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 11 140311 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Motion Analysis Motivation Differential Motion Optical
More informationMotion Estimation. There are three main types (or applications) of motion estimation:
Members: D91922016 朱威達 R93922010 林聖凱 R93922044 謝俊瑋 Motion Estimation There are three main types (or applications) of motion estimation: Parametric motion (image alignment) The main idea of parametric motion
More informationOptical Flow-Based Motion Estimation. Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides.
Optical Flow-Based Motion Estimation Thanks to Steve Seitz, Simon Baker, Takeo Kanade, and anyone else who helped develop these slides. 1 Why estimate motion? We live in a 4-D world Wide applications Object
More informationLocal Image Registration: An Adaptive Filtering Framework
Local Image Registration: An Adaptive Filtering Framework Gulcin Caner a,a.murattekalp a,b, Gaurav Sharma a and Wendi Heinzelman a a Electrical and Computer Engineering Dept.,University of Rochester, Rochester,
More informationCamera Model and Calibration. Lecture-12
Camera Model and Calibration Lecture-12 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationCapturing, Modeling, Rendering 3D Structures
Computer Vision Approach Capturing, Modeling, Rendering 3D Structures Calculate pixel correspondences and extract geometry Not robust Difficult to acquire illumination effects, e.g. specular highlights
More informationMulti-stable Perception. Necker Cube
Multi-stable Perception Necker Cube Spinning dancer illusion, Nobuyuki Kayahara Multiple view geometry Stereo vision Epipolar geometry Lowe Hartley and Zisserman Depth map extraction Essential matrix
More informationComputer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing
More informationNinio, J. and Stevens, K. A. (2000) Variations on the Hermann grid: an extinction illusion. Perception, 29,
Ninio, J. and Stevens, K. A. (2000) Variations on the Hermann grid: an extinction illusion. Perception, 29, 1209-1217. CS 4495 Computer Vision A. Bobick Sparse to Dense Correspodence Building Rome in
More informationCS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka
CS223b Midterm Exam, Computer Vision Monday February 25th, Winter 2008, Prof. Jana Kosecka Your name email This exam is 8 pages long including cover page. Make sure your exam is not missing any pages.
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationMarcel Worring Intelligent Sensory Information Systems
Marcel Worring worring@science.uva.nl Intelligent Sensory Information Systems University of Amsterdam Information and Communication Technology archives of documentaries, film, or training material, video
More informationCS231A Section 6: Problem Set 3
CS231A Section 6: Problem Set 3 Kevin Wong Review 6 -! 1 11/09/2012 Announcements PS3 Due 2:15pm Tuesday, Nov 13 Extra Office Hours: Friday 6 8pm Huang Common Area, Basement Level. Review 6 -! 2 Topics
More informationRange Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
More informationComparison Between The Optical Flow Computational Techniques
Comparison Between The Optical Flow Computational Techniques Sri Devi Thota #1, Kanaka Sunanda Vemulapalli* 2, Kartheek Chintalapati* 3, Phanindra Sai Srinivas Gudipudi* 4 # Associate Professor, Dept.
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics
More informationVisual Tracking (1) Tracking of Feature Points and Planar Rigid Objects
Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
More informationAssignment: Backgrounding and Optical Flow.
Assignment: Backgrounding and Optical Flow. April 6, 00 Backgrounding In this part of the assignment, you will develop a simple background subtraction program.. In this assignment, you are given two videos.
More informationMariya Zhariy. Uttendorf Introduction to Optical Flow. Mariya Zhariy. Introduction. Determining. Optical Flow. Results. Motivation Definition
to Constraint to Uttendorf 2005 Contents to Constraint 1 Contents to Constraint 1 2 Constraint Contents to Constraint 1 2 Constraint 3 Visual cranial reflex(vcr)(?) to Constraint Rapidly changing scene
More informationModule 7 VIDEO CODING AND MOTION ESTIMATION
Module 7 VIDEO CODING AND MOTION ESTIMATION Lesson 20 Basic Building Blocks & Temporal Redundancy Instructional Objectives At the end of this lesson, the students should be able to: 1. Name at least five
More informationTime-to-Contact from Image Intensity
Time-to-Contact from Image Intensity Yukitoshi Watanabe Fumihiko Sakaue Jun Sato Nagoya Institute of Technology Gokiso, Showa, Nagoya, 466-8555, Japan {yukitoshi@cv.,sakaue@,junsato@}nitech.ac.jp Abstract
More informationComputer Vision Lecture 20
Computer Vision Lecture 2 Motion and Optical Flow Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de 28.1.216 Man slides adapted from K. Grauman, S. Seitz, R. Szeliski,
More informationGlobal Flow Estimation. Lecture 9
Motion Models Image Transformations to relate two images 3D Rigid motion Perspective & Orthographic Transformation Planar Scene Assumption Transformations Translation Rotation Rigid Affine Homography Pseudo
More informationCamera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationEXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006,
School of Computer Science and Communication, KTH Danica Kragic EXAM SOLUTIONS Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, 14.00 19.00 Grade table 0-25 U 26-35 3 36-45
More information1 (5 max) 2 (10 max) 3 (20 max) 4 (30 max) 5 (10 max) 6 (15 extra max) total (75 max + 15 extra)
Mierm Exam CS223b Stanford CS223b Computer Vision, Winter 2004 Feb. 18, 2004 Full Name: Email: This exam has 7 pages. Make sure your exam is not missing any sheets, and write your name on every page. The
More informationHorn-Schunck and Lucas Kanade 1
Horn-Schunck and Lucas Kanade 1 Lecture 8 See Sections 4.2 and 4.3 in Reinhard Klette: Concise Computer Vision Springer-Verlag, London, 2014 1 See last slide for copyright information. 1 / 40 Where We
More informationFundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F
Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental
More informationFinal Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
More informationVisual Tracking. Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania.
Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania 1 What is visual tracking? estimation of the target location over time 2 applications Six main areas:
More informationThe 2D/3D Differential Optical Flow
The 2D/3D Differential Optical Flow Prof. John Barron Dept. of Computer Science University of Western Ontario London, Ontario, Canada, N6A 5B7 Email: barron@csd.uwo.ca Phone: 519-661-2111 x86896 Canadian
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationSpatial track: motion modeling
Spatial track: motion modeling Virginio Cantoni Computer Vision and Multimedia Lab Università di Pavia Via A. Ferrata 1, 27100 Pavia virginio.cantoni@unipv.it http://vision.unipv.it/va 1 Comparison between
More informationCS4495 Fall 2014 Computer Vision Problem Set 5: Optic Flow
CS4495 Fall 2014 Computer Vision Problem Set 5: Optic Flow DUE: Wednesday November 12-11:55pm In class we discussed optic flow as the problem of computing a dense flow field where a flow field is a vector
More informationMASTER THESIS. Optical Flow Features for Event Detection. Mohammad Afrooz Mehr, Maziar Haghpanah
Master's Thesis in Embedded and Intelligent Systems, 120 credits MASTER THESIS Optical Flow Features for Event Detection Mohammad Afrooz Mehr, Maziar Haghpanah Embedded and Intelligent Systems, 120 credits
More informationChapter 7: Computation of the Camera Matrix P
Chapter 7: Computation of the Camera Matrix P Arco Nederveen Eagle Vision March 18, 2008 Arco Nederveen (Eagle Vision) The Camera Matrix P March 18, 2008 1 / 25 1 Chapter 7: Computation of the camera Matrix
More informationThe Lucas & Kanade Algorithm
The Lucas & Kanade Algorithm Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Registration, Registration, Registration. Linearizing Registration. Lucas & Kanade Algorithm. 3 Biggest
More informationCPSC 425: Computer Vision
1 / 49 CPSC 425: Computer Vision Instructor: Fred Tung ftung@cs.ubc.ca Department of Computer Science University of British Columbia Lecture Notes 2015/2016 Term 2 2 / 49 Menu March 10, 2016 Topics: Motion
More informationMassachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision QUIZ II
Massachusetts Institute of Technology Department of Computer Science and Electrical Engineering 6.801/6.866 Machine Vision QUIZ II Handed out: 001 Nov. 30th Due on: 001 Dec. 10th Problem 1: (a (b Interior
More informationChapter 18. Geometric Operations
Chapter 18 Geometric Operations To this point, the image processing operations have computed the gray value (digital count) of the output image pixel based on the gray values of one or more input pixels;
More informationCamera model and multiple view geometry
Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then
More informationAutomatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Today Go over Midterm Go over Project #3
More informationFinal Exam Study Guide CSE/EE 486 Fall 2007
Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.
More informationCOMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION
COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION Mr.V.SRINIVASA RAO 1 Prof.A.SATYA KALYAN 2 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING PRASAD V POTLURI SIDDHARTHA
More informationVisual Tracking. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania
Visual Tracking Antonino Furnari Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania furnari@dmi.unict.it 11 giugno 2015 What is visual tracking? estimation
More informationIntroduction to Homogeneous coordinates
Last class we considered smooth translations and rotations of the camera coordinate system and the resulting motions of points in the image projection plane. These two transformations were expressed mathematically
More informationGlobal Flow Estimation. Lecture 9
Global Flow Estimation Lecture 9 Global Motion Estimate motion using all pixels in the image. Parametric flow gives an equation, which describes optical flow for each pixel. Affine Projective Global motion
More informationMotion. CS 554 Computer Vision Pinar Duygulu Bilkent University
1 Motion CS 554 Computer Vision Pinar Duygulu Bilkent University 2 Motion A lot of information can be extracted from time varying sequences of images, often more easily than from static images. e.g, camouflaged
More informationAn idea which can be used once is a trick. If it can be used more than once it becomes a method
An idea which can be used once is a trick. If it can be used more than once it becomes a method - George Polya and Gabor Szego University of Texas at Arlington Rigid Body Transformations & Generalized
More informationStereo Observation Models
Stereo Observation Models Gabe Sibley June 16, 2003 Abstract This technical report describes general stereo vision triangulation and linearized error modeling. 0.1 Standard Model Equations If the relative
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Oct 2009 Motion estimation Goals Motion estimation Affine flow Optimization Large motions Why affine? Monday dense, smooth motion and regularization. Robust
More informationSpatial track: motion modeling
Spatial track: motion modeling Virginio Cantoni Computer Vision and Multimedia Lab Università di Pavia Via A. Ferrata 1, 27100 Pavia virginio.cantoni@unipv.it http://vision.unipv.it/va 1 Comparison between
More informationTHE VIEWING TRANSFORMATION
ECS 178 Course Notes THE VIEWING TRANSFORMATION Kenneth I. Joy Institute for Data Analysis and Visualization Department of Computer Science University of California, Davis Overview One of the most important
More informationHand-Eye Calibration from Image Derivatives
Hand-Eye Calibration from Image Derivatives Abstract In this paper it is shown how to perform hand-eye calibration using only the normal flow field and knowledge about the motion of the hand. The proposed
More informationComputational Optical Imaging - Optique Numerique. -- Single and Multiple View Geometry, Stereo matching --
Computational Optical Imaging - Optique Numerique -- Single and Multiple View Geometry, Stereo matching -- Autumn 2015 Ivo Ihrke with slides by Thorsten Thormaehlen Reminder: Feature Detection and Matching
More informationOptic Flow and Motion Detection
Optic Flow and Motion Detection Computer Vision and Imaging Martin Jagersand Readings: Szeliski Ch 8 Ma, Kosecka, Sastry Ch 4 Image motion Somehow quantify the frame-to to-frame differences in image sequences.
More informationAnnouncements. Motion. Structure-from-Motion (SFM) Motion. Discrete Motion: Some Counting
Announcements Motion Introduction to Computer Vision CSE 152 Lecture 20 HW 4 due Friday at Midnight Final Exam: Tuesday, 6/12 at 8:00AM-11:00AM, regular classroom Extra Office Hours: Monday 6/11 9:00AM-10:00AM
More informationCSE 252A Computer Vision Homework 3 Instructor: Ben Ochoa Due : Monday, November 21, 2016, 11:59 PM
CSE 252A Computer Vision Homework 3 Instructor: Ben Ochoa Due : Monday, November 21, 2016, 11:59 PM Instructions: Homework 3 has to be submitted in groups of 3. Review the academic integrity and collaboration
More informationSIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image
More informationROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW
ROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW Thorsten Thormählen, Hellward Broszio, Ingolf Wassermann thormae@tnt.uni-hannover.de University of Hannover, Information Technology Laboratory,
More informationextracted occurring from the spatial and temporal changes in an image sequence. An image sequence
Motion: Introduction are interested in the visual information that can be We from the spatial and temporal changes extracted in an image sequence. An image sequence occurring of a series of images (frames)
More informationStructure from Motion. Prof. Marco Marcon
Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)
More information