Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration"

Transcription

1 Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1 st : F. Zafar, Y. Wang, and N. Chhaya! Dec 6 th : J. Dandois! Dec 8 th :! Dec 13 th :! Dec 20 th : +5 bonus points +3 bonus points +2 bonus points +1 bonus points +0 bonus points +0 bonus points Note: You have until next Monday to let me know

2 Camera Calibration: Motivation! Images are obtained from 3D scenes! The exact position and orientation of the camera sensing device are often unknown! Camera needs to be related to some global frame / reference to be able to:! Get accurate measurements! Inspect materials! Etc Camera Model! Camera model:! from world to camera (coordinate systems)! from camera (3D) to image frame (2D) x im (x im,y im ) O y x y im p P P w Z w X w Y w

3 Camera Calibration! Goal: build the geometric relations between a 3D scene and its 2D images.! 3D transformation that represents the viewpoints and viewing directions of the camera! The perspective projection that maps 3D points into 2D images Camera Model: Parameters! Camera Parameters! Intrinsic Parameters (of the camera): link the frame coordinates of an image point with its corresponding camera coordinates! Extrinsic parameters: define the location and orientation of the camera coordinate system with respect to the world coordinate system x im (x im,y im ) O y x y im p P P w Z w X w Y w

4 Transformations 2-D Rotation (x, y ) (x, y)! x = x cos(!) - y sin(!) y = x sin(!) + y cos(!)

5 2D Rotation! Rotation by angle! (about the origin)!"#"%!"#"%!% &()%*+%),%*-.,/+,0% 12/%/2)(32-+4% Scaling! Scaling operation:! Or, in matrix form: " x % " = a 0 %" # y & # 0 b& x % # y & scaling matrix S

6 Scaling! Scaling a coordinate means multiplying each of its components by a scalar! Uniform scaling means this scalar is the same for all components: " 2 Scaling! Uniform scaling by s:!"#"%!"#"%

7 Non-uniform Scaling! Non-uniform scaling: different scalars per component: X " 2, Y " 0.5 2x2 Matrices! What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)?

8 All 2D Linear Transformations! Linear transformations are combinations of! Scale,! Rotation,! Shear, and! Mirror! Properties of linear transformations:! Origin maps to origin! Lines map to lines! Parallel lines remain parallel! Ratios are preserved! Closed under composition What about 2D Translation? 56%7/(-+8(32-0%! Translation is not a liner operation on 2D coordinates! Homogeneous coordinates

9 Homogeneous coordinates 7/*9:4%%(;;%2-,%<2/,%922/;*-(),4% B%!%#%&#%% 2<2=,-,2>+%F8(-,% 2<2=,-,2>+%*<(=,%% 922/;*-(),+% A% Homogeneous Coordinates! Add a 3rd coordinate to every 2D point! (x, y, w) represents a point at location (x/w, y/w)! (x, y, 0) represents a point at infinity! (0, 0, 0) is not allowed y 2 (2,1,1) or (4,2,2) or (6,3,3) Convenient coordinate 1 system to represent many useful x transformations 1 2

10 2D Translation?! Solution: homogeneous coordinates to the rescue Translation! Example of translation Homogeneous Coordinates " x % " 1 0 t x %" x% y = 0 1 t y y # 1 & # & # 1& " x + t x % = y + t y # 1 & t x = 2 t y = 1

11 Affine transformations (-A%)/(-+G2/<(32-%B*)%% 8(+)%/2B%H%"%"%D%I%B,%9(88%(-%% )*+,%)/(-+G2/<(32-- Basic affine transformations Translate Scale 2D in-plane rotation Shear

12 3D Rotations & ) X " = & 0 cos" #sin") %& 0 sin" cos" () cos" 0 sin" & ) Y " = & ) %& #sin" 0 cos" () cos" #sin" 0 & ) Z " = & sin" cos" 0) %& 0 0 1( ) Where do we go from here? (J-,%)/(-+G2/<(32-% B()%(FF,-+%B,-%B,% 9(-=,%)*+%/2B0-

13 Projective Transformations?(88,;%(%.##/")0.&%%!2/%01)+)"-0,"20,345,-)0- Image warping with homographies image plane in front image plane below black area where no pixel maps to

14 Camera Calibration Camera Model: Parameters! Camera Parameters! Intrinsic Parameters (of the camera): link the frame coordinates of an image point with its corresponding camera coordinates! Extrinsic parameters: define the location and orientation of the camera coordinate system with respect to the world coordinate system x im (x im,y im ) O y x y im p P P w Z w X w Y w

15 Camera coordinate system Principal axis: line from the camera center perpendicular to the image plane Principal point (p): point where principal axis intersects the image plane (origin of normalized coordinate system) Perspective Projection Equations (X-Z Plane) X x f Z "x f = X Z "x = f X Z Notice: Position on the image plane is related to depth. (u,v) are scaled equally based on focal length and depth

16 Equivalent Image Geometries! Consider case with object on optical axis! More convenient geometry, with an upright image! Both are equivalent mathematically x = f X Z (X,Y,Z)! ( f X Z, f Y Z ) X Y Z 1 " # % &! f X f Y Z " # % & Pinhole camera model X Y Z 1 " # % & f 0 f " # % & =

17 Principal point offset principal point: Camera coordinate system: origin is at the principal point Image coordinate system: origin is in the corner Principal point offset principal point: (X,Y,Z)! ( f X Z + p x, f Y Z + p y )

18 Pixel size Pixel size:! m x pixels per meter in horizontal direction, m y pixels per meter in vertical direction pixels/m m pixels Skew The camera coordinate system may be skewed due to some manufacturing error.

19 Lens distortion! Two main lens distortion:! Radial distortion: arise as a result of the shape of lens (e.g. fish-eye)! Tangential distortion: arise from the assembly process of the camera as a whole. Radial Distortion! Zero at the optical center of the image! Increases as we move toward the periphery! Rays farther from the center of the lens are bent more than those closer in! Terms: k1, k2, k3

20 Intrinsic parameters! Principal point coordinates! Focal length! Pixel magnification factors! Skew (non-rectangular pixels)! Radial distortion Camera parameters Extrinsic parameters! Rotation and translation relative to world coordinate system Camera rotation and translation In general, the camera coordinate frame will be related to the world coordinate frame by a rotation and a translation ( ) X cam = R X - C coords. of point in camera frame coords. of a point in world frame (nonhomogeneous) coords. of camera center in world frame

21 Intrinsic parameters! Principal point coordinates! Focal length! Pixel magnification factors! Skew (non-rectangular pixels)! Radial distortion Camera parameters Extrinsic parameters! Rotation and translation relative to world coordinate system Camera Calibration Basic matrix: " % " t 1 %" R 11 R 12 R 13 0% " T 1 % G = PLRT = t 2 R 21 R 22 R T t 3 R 31 R 32 R T 3 # 0 0 1/ f 0& # &# & # & Intrinsic Parameters Extrinsic Parameters How to estimate those parameters?

22 Camera calibration Given n points with known 3D coordinates X i and known image projections x i, estimate the camera parameters X i x i Example Calibration Pattern Harris Corner Detector From Sebastian Thrun 44

23 Camera calibration Calibration: 2 steps! Step 1: Transform into camera coordinates " X C % " X W % Y C # Z C = f ( Y W & Z W,(,),*,T) # &! Step 2: Transform into image coordinates 46

24 Calibration Model (extrinsic) 47 Advantage of Homogeneous C s i-th data point 48

25 Camera Calibration! Algorithm 1. Measure N 3D coordinates (Xi, Yi,Zi) 2. Locate their corresponding image points (xi,yi) - Edge, Corner, Hough 3. Build matrix A of a homogeneous system Av = 0 4. Compute SVD of A, solution v 5. Determine aspect ratio # and scale 6. Recover the first two rows of R and the first two components of T up to a sign 7. Determine sign s of by checking the projection equation 8. Compute the 3 rd row of R by vector product, and enforce orthogonality constraint by SVD 9. Solve Tz and fx using Least Square and SVD, then fy = fx / #% Y w Z w X w How Many Images Do We Need?! Assumption: K images with M corners each! 4+6K parameters! 2KM constraints! 2KM & 4+6K M>3 and K &2/(M-3)! 2 images with 4 points, but will 1 images with 5 points work?! No, since points cannot be co-planar! 50

26 Calibration tools? Multi-view Geometry

27 Two-view geometry Scene geometry (structure): Given corresponding points in two or more images, where is the pre-image of these points in 3D? Correspondence (stereo matching): Given a point in just one image, how does it constrain the position of the corresponding point x in another image? Camera geometry (motion): Given a set of corresponding points in two images, what are the cameras for the two views? Parameters of a Stereo System! Intrinsic Parameters! Characterize the transformation from camera to pixel coordinate systems of each camera! Focal length, image center, aspect ratio P l P P r! Extrinsic parameters! Describe the relative position and orientation of the two cameras X l! Rotation matrix R and translation vector T p l Y l f l O l Z l R, T Z r X r p r Y r f r O r Bahadir K. Gunturk

28 Triangulation Given projections of a 3D point in two or more images (with known camera matrices), find the coordinates of the point X? x 1 x 2 O 1 O 2 Triangulation We want to intersect the two visual rays corresponding to x 1 and x 2, but because of noise and numerical errors, they don t meet exactly R 2 X? R 1 x 1 x 2 O 1 O 2

29 Triangulation: Geometric approach Find shortest segment connecting the two viewing rays and let X be the midpoint of that segment X x 1 x 2 O 1 O 2 Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the other camera center = vanishing points of camera motion direction Epipolar Lines - intersections of epipolar plane with image planes (always come in corresponding pairs)

30 Epipolar constraint X x x If we observe a point x in one image, where can the corresponding point x be in the other image? Epipolar constraint X X X x x x x Potential matches for x have to lie on the corresponding epipolar line l. Potential matches for x have to lie on the corresponding epipolar line l.

31 Calibrated Camera Essential matrix Bahadir K. Gunturk Example: Converging cameras

32 Rectification! Problem: Epipolar lines not parallel to scan lines P P l P r Epipolar Plane p l Epipolar Lines p r O l e l e r Or Epipoles From Sebastian Thrun/Jana Kosecka Rectification! Problem: Epipolar lines not parallel to scan lines P Epipolar Plane P l P r Rectified Images p l Epipolar Lines p r O l Or Epipoles at infinity From Sebastian Thrun/Jana Kosecka

33 Epipolar rectification Rectified Image Pair Epipolar rectification Rectified Image Pair

34 Stereo results Scene Ground truth (from Seitz) Results with window correlation Window-based matching (best window size) Ground truth (from Seitz)

35 Results with better method State of the art Ground truth Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September (from Seitz) How can We Improve Stereo? Space-time stereo scanner uses unstructured light to aid in correspondence Result: Dense 3D mesh (noisy) From Sebastian Thrun/Jana Kosecka

36 Unstructured Light 3D Scanner OpenCV! cvfindchessboardcorners()! Calibrate:! cvcalibratecamera2()! cvstereocalibrate()! Rectify / Epipolar! cvinitundistortmap()! cvinitundistortrectifymap()! cvcomputecorrespondepilines()! cvfindfundamentalmatrix()! cvstereorectify()! Stereo! cvfindstereocorrespondencebm()! Depth maps! cvreprojectimageto3d()

37 Assignment #4! Download the Matlab Camera Calibration Toolbox! Read Documentation! First calibration example! Download data! Run camera calibration software! Submit results Conclusion! There s a mathematical model to describe cameras! Parameters can be estimated using a camera calibration approach! Camera calibration is key for accurate stereo reconstruction

Pin Hole Cameras & Warp Functions

Pin Hole Cameras & Warp Functions Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Motivation Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg

More information

Computer Vision Projective Geometry and Calibration. Pinhole cameras

Computer Vision Projective Geometry and Calibration. Pinhole cameras Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole

More information

3D Geometry and Camera Calibration

3D Geometry and Camera Calibration 3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often

More information

Camera Model and Calibration

Camera Model and Calibration Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the

More information

Lecture 9: Epipolar Geometry

Lecture 9: Epipolar Geometry Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2

More information

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1)

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture

More information

Lecture 14: Basic Multi-View Geometry

Lecture 14: Basic Multi-View Geometry Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram

More information

CV: 3D to 2D mathematics. Perspective transformation; camera calibration; stereo computation; and more

CV: 3D to 2D mathematics. Perspective transformation; camera calibration; stereo computation; and more CV: 3D to 2D mathematics Perspective transformation; camera calibration; stereo computation; and more Roadmap of topics n Review perspective transformation n Camera calibration n Stereo methods n Structured

More information

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few... STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own

More information

Reminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches

Reminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary

More information

Camera Model and Calibration. Lecture-12

Camera Model and Calibration. Lecture-12 Camera Model and Calibration Lecture-12 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the

More information

Rectification and Disparity

Rectification and Disparity Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide

More information

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah Camera Models and Image Formation Srikumar Ramalingam School of Computing University of Utah srikumar@cs.utah.edu Reference Most slides are adapted from the following notes: Some lecture notes on geometric

More information

Computer Vision. Coordinates. Prof. Flávio Cardeal DECOM / CEFET- MG.

Computer Vision. Coordinates. Prof. Flávio Cardeal DECOM / CEFET- MG. Computer Vision Coordinates Prof. Flávio Cardeal DECOM / CEFET- MG cardeal@decom.cefetmg.br Abstract This lecture discusses world coordinates and homogeneous coordinates, as well as provides an overview

More information

Lecture 9 & 10: Stereo Vision

Lecture 9 & 10: Stereo Vision Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving

More information

Rectification. Dr. Gerhard Roth

Rectification. Dr. Gerhard Roth Rectification Dr. Gerhard Roth Problem Definition Given a pair of stereo images, the intrinsic parameters of each camera, and the extrinsic parameters of the system, R, and, compute the image transformation

More information

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah Camera Models and Image Formation Srikumar Ramalingam School of Computing University of Utah srikumar@cs.utah.edu VisualFunHouse.com 3D Street Art Image courtesy: Julian Beaver (VisualFunHouse.com) 3D

More information

Epipolar geometry. x x

Epipolar geometry. x x Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections

More information

Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy

Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy 1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:

More information

Recovering structure from a single view Pinhole perspective projection

Recovering structure from a single view Pinhole perspective projection EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their

More information

Introduction to Homogeneous coordinates

Introduction to Homogeneous coordinates Last class we considered smooth translations and rotations of the camera coordinate system and the resulting motions of points in the image projection plane. These two transformations were expressed mathematically

More information

Lecture 6 Stereo Systems Multi-view geometry

Lecture 6 Stereo Systems Multi-view geometry Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems

More information

Midterm Exam Solutions

Midterm Exam Solutions Midterm Exam Solutions Computer Vision (J. Košecká) October 27, 2009 HONOR SYSTEM: This examination is strictly individual. You are not allowed to talk, discuss, exchange solutions, etc., with other fellow

More information

Projective Geometry and Camera Models

Projective Geometry and Camera Models /2/ Projective Geometry and Camera Models Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Note about HW Out before next Tues Prob: covered today, Tues Prob2: covered next Thurs Prob3:

More information

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004 Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,

More information

Projective Geometry and Camera Models

Projective Geometry and Camera Models Projective Geometry and Camera Models Computer Vision CS 43 Brown James Hays Slides from Derek Hoiem, Alexei Efros, Steve Seitz, and David Forsyth Administrative Stuff My Office hours, CIT 375 Monday and

More information

Laser sensors. Transmitter. Receiver. Basilio Bona ROBOTICA 03CFIOR

Laser sensors. Transmitter. Receiver. Basilio Bona ROBOTICA 03CFIOR Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The receiver measures the time of flight (back and

More information

3D Vision Real Objects, Real Cameras. Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun,

3D Vision Real Objects, Real Cameras. Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun, 3D Vision Real Objects, Real Cameras Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun, anders@cb.uu.se 3D Vision! Philisophy! Image formation " The pinhole camera " Projective

More information

Stereo vision. Many slides adapted from Steve Seitz

Stereo vision. Many slides adapted from Steve Seitz Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is

More information

Module 4F12: Computer Vision and Robotics Solutions to Examples Paper 2

Module 4F12: Computer Vision and Robotics Solutions to Examples Paper 2 Engineering Tripos Part IIB FOURTH YEAR Module 4F2: Computer Vision and Robotics Solutions to Examples Paper 2. Perspective projection and vanishing points (a) Consider a line in 3D space, defined in camera-centered

More information

Models and The Viewing Pipeline. Jian Huang CS456

Models and The Viewing Pipeline. Jian Huang CS456 Models and The Viewing Pipeline Jian Huang CS456 Vertex coordinates list, polygon table and (maybe) edge table Auxiliary: Per vertex normal Neighborhood information, arranged with regard to vertices and

More information

3D FACE RECONSTRUCTION BASED ON EPIPOLAR GEOMETRY

3D FACE RECONSTRUCTION BASED ON EPIPOLAR GEOMETRY IJDW Volume 4 Number January-June 202 pp. 45-50 3D FACE RECONSRUCION BASED ON EPIPOLAR GEOMERY aher Khadhraoui, Faouzi Benzarti 2 and Hamid Amiri 3,2,3 Signal, Image Processing and Patterns Recognition

More information

Perspective Projection [2 pts]

Perspective Projection [2 pts] Instructions: CSE252a Computer Vision Assignment 1 Instructor: Ben Ochoa Due: Thursday, October 23, 11:59 PM Submit your assignment electronically by email to iskwak+252a@cs.ucsd.edu with the subject line

More information

Multi-view geometry problems

Multi-view geometry problems Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera

More information

3D Viewing. CS 4620 Lecture 8

3D Viewing. CS 4620 Lecture 8 3D Viewing CS 46 Lecture 8 13 Steve Marschner 1 Viewing, backward and forward So far have used the backward approach to viewing start from pixel ask what part of scene projects to pixel explicitly construct

More information

CS 231A: Computer Vision (Winter 2018) Problem Set 2

CS 231A: Computer Vision (Winter 2018) Problem Set 2 CS 231A: Computer Vision (Winter 2018) Problem Set 2 Due Date: Feb 09 2018, 11:59pm Note: In this PS, using python2 is recommended, as the data files are dumped with python2. Using python3 might cause

More information

Lecture 3: Camera Calibration, DLT, SVD

Lecture 3: Camera Calibration, DLT, SVD Computer Vision Lecture 3 23--28 Lecture 3: Camera Calibration, DL, SVD he Inner Parameters In this section we will introduce the inner parameters of the cameras Recall from the camera equations λx = P

More information

Image Formation I Chapter 2 (R. Szelisky)

Image Formation I Chapter 2 (R. Szelisky) Image Formation I Chapter 2 (R. Selisky) Guido Gerig CS 632 Spring 22 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified from

More information

Visual Recognition: Image Formation

Visual Recognition: Image Formation Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know

More information

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes

More information

Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision

Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching

More information

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science. Professor Willia Hoff Dept of Electrical Engineering &Coputer Science http://inside.ines.edu/~whoff/ 1 Caera Calibration 2 Caera Calibration Needed for ost achine vision and photograetry tasks (object

More information

Lecture 7 Measurement Using a Single Camera. Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016

Lecture 7 Measurement Using a Single Camera. Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016 Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016 If I have an image containing a coin, can you tell me the diameter of that coin? Contents

More information

Compositing a bird's eye view mosaic

Compositing a bird's eye view mosaic Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition

More information

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix

More information

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views? Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple

More information

Final Exam Study Guide CSE/EE 486 Fall 2007

Final Exam Study Guide CSE/EE 486 Fall 2007 Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.

More information

Structure from Motion. Prof. Marco Marcon

Structure from Motion. Prof. Marco Marcon Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)

More information

Agenda. Perspective projection. Rotations. Camera models

Agenda. Perspective projection. Rotations. Camera models Image formation Agenda Perspective projection Rotations Camera models Light as a wave + particle Light as a wave (ignore for now) Refraction Diffraction Image formation Digital Image Film Human eye Pixel

More information

Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras

Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/ Outline The geometry of active stereo.

More information

Scalable geometric calibration for multi-view camera arrays

Scalable geometric calibration for multi-view camera arrays Scalable geometric calibration for multi-view camera arrays Bernhard Blaschitz, Doris Antensteiner, Svorad Štolc Bernhard.Blaschitz@ait.ac.at AIT Austrian Institute of Technology GmbH Intelligent Vision

More information

Basilio Bona DAUIN Politecnico di Torino

Basilio Bona DAUIN Politecnico di Torino ROBOTICA 03CFIOR DAUIN Politecnico di Torino Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The

More information

Single View Geometry. Camera model & Orientation + Position estimation. What am I?

Single View Geometry. Camera model & Orientation + Position estimation. What am I? Single View Geometry Camera model & Orientation + Position estimation What am I? Vanishing points & line http://www.wetcanvas.com/ http://pennpaint.blogspot.com/ http://www.joshuanava.biz/perspective/in-other-words-the-observer-simply-points-in-thesame-direction-as-the-lines-in-order-to-find-their-vanishing-point.html

More information

Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15

Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems

More information

Geometry of image formation

Geometry of image formation eometry of image formation Tomáš Svoboda, svoboda@cmp.felk.cvut.cz Czech Technical University in Prague, Center for Machine Perception http://cmp.felk.cvut.cz Last update: November 3, 2008 Talk Outline

More information

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix

More information

Projector Calibration for Pattern Projection Systems

Projector Calibration for Pattern Projection Systems Projector Calibration for Pattern Projection Systems I. Din *1, H. Anwar 2, I. Syed 1, H. Zafar 3, L. Hasan 3 1 Department of Electronics Engineering, Incheon National University, Incheon, South Korea.

More information

Depth from two cameras: stereopsis

Depth from two cameras: stereopsis Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture

More information

Stereo Vision Based Mapping and Immediate Virtual Walkthroughs

Stereo Vision Based Mapping and Immediate Virtual Walkthroughs Stereo Vision Based Mapping and Immediate Virtual Walkthroughs Heiko Hirschmüller M.Sc., Dipl.-Inform. (FH) Submitted in partial fulfilment of the requirements for the degree of DOCTOR OF PHILOSOPHY June

More information

Camera Calibration using Vanishing Points

Camera Calibration using Vanishing Points Camera Calibration using Vanishing Points Paul Beardsley and David Murray * Department of Engineering Science, University of Oxford, Oxford 0X1 3PJ, UK Abstract This paper describes a methodformeasuringthe

More information

Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds

Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds Marta Wilczkowiak Edmond Boyer Peter Sturm Movi Gravir Inria Rhône-Alpes, 655 Avenue de l Europe, 3833 Montbonnot, France

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

More information

Three-Dimensional Viewing Hearn & Baker Chapter 7

Three-Dimensional Viewing Hearn & Baker Chapter 7 Three-Dimensional Viewing Hearn & Baker Chapter 7 Overview 3D viewing involves some tasks that are not present in 2D viewing: Projection, Visibility checks, Lighting effects, etc. Overview First, set up

More information

Image Warping and Mosacing

Image Warping and Mosacing Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

More information

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking. In close-up photo, the depth of field is limited. Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

More information

Binocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?

Binocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from? Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo

More information

Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the

Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the corresponding 3D points. The projection models include:

More information

1 CSE 252A Computer Vision I Fall 2017

1 CSE 252A Computer Vision I Fall 2017 Assignment 1 CSE A Computer Vision I Fall 01 1.1 Assignment This assignment contains theoretical and programming exercises. If you plan to submit hand written answers for theoretical exercises, please

More information

CSE 167: Introduction to Computer Graphics Lecture #3: Coordinate Systems

CSE 167: Introduction to Computer Graphics Lecture #3: Coordinate Systems CSE 167: Introduction to Computer Graphics Lecture #3: Coordinate Systems Jürgen P. Schulze, Ph.D. University of California, San Diego Spring Quarter 2015 Announcements Project 2 due Friday at 1pm Homework

More information

Introduction to Computer Vision. Introduction CMPSCI 591A/691A CMPSCI 570/670. Image Formation

Introduction to Computer Vision. Introduction CMPSCI 591A/691A CMPSCI 570/670. Image Formation Introduction CMPSCI 591A/691A CMPSCI 570/670 Image Formation Lecture Outline Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic

More information

Jump Stitch Metadata & Depth Maps Version 1.1

Jump Stitch Metadata & Depth Maps Version 1.1 Jump Stitch Metadata & Depth Maps Version 1.1 jump-help@google.com Contents 1. Introduction 1 2. Stitch Metadata File Format 2 3. Coverage Near the Poles 4 4. Coordinate Systems 6 5. Camera Model 6 6.

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

More information

Elements of Computer Vision: Multiple View Geometry. 1 Introduction. 2 Elements of Geometry. Andrea Fusiello

Elements of Computer Vision: Multiple View Geometry. 1 Introduction. 2 Elements of Geometry. Andrea Fusiello Elements of Computer Vision: Multiple View Geometry. Andrea Fusiello http://www.sci.univr.it/~fusiello July 11, 2005 c Copyright by Andrea Fusiello. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike

More information

Transforms. COMP 575/770 Spring 2013

Transforms. COMP 575/770 Spring 2013 Transforms COMP 575/770 Spring 2013 Transforming Geometry Given any set of points S Could be a 2D shape, a 3D object A transform is a function T that modifies all points in S: T S S T v v S Different transforms

More information

Single-view metrology

Single-view metrology Single-view metrology Magritte, Personal Values, 952 Many slides from S. Seitz, D. Hoiem Camera calibration revisited What if world coordinates of reference 3D points are not known? We can use scene features

More information

Overview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers

Overview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented

More information

A Stratified Approach for Camera Calibration Using Spheres

A Stratified Approach for Camera Calibration Using Spheres IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, MONTH YEAR 1 A Stratified Approach for Camera Calibration Using Spheres Kwan-Yee K. Wong, Member, IEEE, Guoqiang Zhang, Student-Member, IEEE and Zhihu

More information

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science. Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Stereo Vision 2 Inferring 3D from 2D Model based pose estimation single (calibrated) camera Stereo

More information

Vision 3D articielle Multiple view geometry

Vision 3D articielle Multiple view geometry Vision 3D articielle Multiple view geometry Pascal Monasse monasse@imagine.enpc.fr IMAGINE, École des Ponts ParisTech Contents Multi-view constraints Multi-view calibration Incremental calibration Global

More information

Rectification for Any Epipolar Geometry

Rectification for Any Epipolar Geometry Rectification for Any Epipolar Geometry Daniel Oram Advanced Interfaces Group Department of Computer Science University of Manchester Mancester, M13, UK oramd@cs.man.ac.uk Abstract This paper proposes

More information

AUTO-RECTIFICATION OF USER PHOTOS. Krishnendu Chaudhury, Stephen DiVerdi, Sergey Ioffe.

AUTO-RECTIFICATION OF USER PHOTOS. Krishnendu Chaudhury, Stephen DiVerdi, Sergey Ioffe. AUTO-RECTIFICATION OF USER PHOTOS Krishnendu Chaudhury, Stephen DiVerdi, Sergey Ioffe krish@google.com, diverdi@google.com, sioffe@google.com ABSTRACT The image auto rectification project at Google aims

More information

Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC

Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC Shuji Sakai, Koichi Ito, Takafumi Aoki Graduate School of Information Sciences, Tohoku University, Sendai, 980 8579, Japan Email: sakai@aoki.ecei.tohoku.ac.jp

More information

Part I: Single and Two View Geometry Internal camera parameters

Part I: Single and Two View Geometry Internal camera parameters !! 43 1!???? Imaging eometry Multiple View eometry Perspective projection Richard Hartley Andrew isserman O p y VPR June 1999 where image plane This can be written as a linear mapping between homogeneous

More information

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller 3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza

Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,

More information

Computer Vision. Geometric Camera Calibration. Samer M Abdallah, PhD

Computer Vision. Geometric Camera Calibration. Samer M Abdallah, PhD Computer Vision Samer M Abdallah, PhD Faculty of Engineering and Architecture American University of Beirut Beirut, Lebanon Geometric Camera Calibration September 2, 2004 1 Computer Vision Geometric Camera

More information

Image warping and stitching

Image warping and stitching Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize

More information

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 4 Affine Structure from Motion Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D, Newmark Civil Engineering

More information

Exterior Orientation Parameters

Exterior Orientation Parameters Exterior Orientation Parameters PERS 12/2001 pp 1321-1332 Karsten Jacobsen, Institute for Photogrammetry and GeoInformation, University of Hannover, Germany The georeference of any photogrammetric product

More information

SRI Small Vision System

SRI Small Vision System Small Vision System Calibration 1 SRI Small Vision System Calibration Supplement to the User s Manual Software version 2.2b July 2001 Kurt Konolige and David Beymer SRI International konolige@ai.sri.com

More information

Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision

Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras

More information

Omni Stereo Vision of Cooperative Mobile Robots

Omni Stereo Vision of Cooperative Mobile Robots Omni Stereo Vision of Cooperative Mobile Robots Zhigang Zhu*, Jizhong Xiao** *Department of Computer Science **Department of Electrical Engineering The City College of the City University of New York (CUNY)

More information

Lecture 5.3 Camera calibration. Thomas Opsahl

Lecture 5.3 Camera calibration. Thomas Opsahl Lecture 5.3 Camera calibration Thomas Opsahl Introduction The image u u x X W v C z C World frame x C y C z C = 1 For finite projective cameras, the correspondence between points in the world and points

More information

Mapping Non-Destructive Testing Data on the 3D Geometry of Objects with Complex Shapes

Mapping Non-Destructive Testing Data on the 3D Geometry of Objects with Complex Shapes More Info at Open Access Database www.ndt.net/?id=17670 Mapping Non-Destructive Testing Data on the 3D Geometry of Objects with Complex Shapes Abstract by S. Soldan*, D. Ouellet**, P.Hedayati**, A. Bendada**,

More information

Perspective Correction Methods for Camera-Based Document Analysis

Perspective Correction Methods for Camera-Based Document Analysis Perspective Correction Methods for Camera-Based Document Analysis L. Jagannathan and C. V. Jawahar Center for Visual Information Technology International Institute of Information Technology Gachibowli,

More information

Robust Geometry Estimation from two Images

Robust Geometry Estimation from two Images Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based

More information

CSE 252B: Computer Vision II

CSE 252B: Computer Vision II CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points

More information

DTU M.SC. - COURSE EXAM Revised Edition

DTU M.SC. - COURSE EXAM Revised Edition Written test, 16 th of December 1999. Course name : 04250 - Digital Image Analysis Aids allowed : All usual aids Weighting : All questions are equally weighed. Name :...................................................

More information

CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein. Lecture 23: Photometric Stereo

CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein. Lecture 23: Photometric Stereo CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein Lecture 23: Photometric Stereo Announcements PA3 Artifact due tonight PA3 Demos Thursday Signups close at 4:30 today No lecture on Friday Last Time:

More information