Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration

Save this PDF as:

Size: px
Start display at page:

Download "Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration"

Transcription

1 Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1 st : F. Zafar, Y. Wang, and N. Chhaya! Dec 6 th : J. Dandois! Dec 8 th :! Dec 13 th :! Dec 20 th : +5 bonus points +3 bonus points +2 bonus points +1 bonus points +0 bonus points +0 bonus points Note: You have until next Monday to let me know

2 Camera Calibration: Motivation! Images are obtained from 3D scenes! The exact position and orientation of the camera sensing device are often unknown! Camera needs to be related to some global frame / reference to be able to:! Get accurate measurements! Inspect materials! Etc Camera Model! Camera model:! from world to camera (coordinate systems)! from camera (3D) to image frame (2D) x im (x im,y im ) O y x y im p P P w Z w X w Y w

3 Camera Calibration! Goal: build the geometric relations between a 3D scene and its 2D images.! 3D transformation that represents the viewpoints and viewing directions of the camera! The perspective projection that maps 3D points into 2D images Camera Model: Parameters! Camera Parameters! Intrinsic Parameters (of the camera): link the frame coordinates of an image point with its corresponding camera coordinates! Extrinsic parameters: define the location and orientation of the camera coordinate system with respect to the world coordinate system x im (x im,y im ) O y x y im p P P w Z w X w Y w

4 Transformations 2-D Rotation (x, y ) (x, y)! x = x cos(!) - y sin(!) y = x sin(!) + y cos(!)

5 2D Rotation! Rotation by angle! (about the origin)!"#"%!"#"%!% &()%*+%),%*-.,/+,0% 12/%/2)(32-+4% Scaling! Scaling operation:! Or, in matrix form: " x % " = a 0 %" # y & # 0 b& x % # y & scaling matrix S

6 Scaling! Scaling a coordinate means multiplying each of its components by a scalar! Uniform scaling means this scalar is the same for all components: " 2 Scaling! Uniform scaling by s:!"#"%!"#"%

7 Non-uniform Scaling! Non-uniform scaling: different scalars per component: X " 2, Y " 0.5 2x2 Matrices! What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Mirror over (0,0)?

8 All 2D Linear Transformations! Linear transformations are combinations of! Scale,! Rotation,! Shear, and! Mirror! Properties of linear transformations:! Origin maps to origin! Lines map to lines! Parallel lines remain parallel! Ratios are preserved! Closed under composition What about 2D Translation? 56%7/(-+8(32-0%! Translation is not a liner operation on 2D coordinates! Homogeneous coordinates

9 Homogeneous coordinates 7/*9:4%%(;;%2-,%<2/,%922/;*-(),4% B%!%#%&#%% 2<2=,-,2>+%F8(-,% 2<2=,-,2>+%*<(=,%% 922/;*-(),+% A% Homogeneous Coordinates! Add a 3rd coordinate to every 2D point! (x, y, w) represents a point at location (x/w, y/w)! (x, y, 0) represents a point at infinity! (0, 0, 0) is not allowed y 2 (2,1,1) or (4,2,2) or (6,3,3) Convenient coordinate 1 system to represent many useful x transformations 1 2

10 2D Translation?! Solution: homogeneous coordinates to the rescue Translation! Example of translation Homogeneous Coordinates " x % " 1 0 t x %" x% y = 0 1 t y y # 1 & # & # 1& " x + t x % = y + t y # 1 & t x = 2 t y = 1

11 Affine transformations (-A%)/(-+G2/<(32-%B*)%% 8(+)%/2B%H%"%"%D%I%B,%9(88%(-%% )*+,%)/(-+G2/<(32-- Basic affine transformations Translate Scale 2D in-plane rotation Shear

12 3D Rotations & ) X " = & 0 cos" #sin") %& 0 sin" cos" () cos" 0 sin" & ) Y " = & ) %& #sin" 0 cos" () cos" #sin" 0 & ) Z " = & sin" cos" 0) %& 0 0 1( ) Where do we go from here? (J-,%)/(-+G2/<(32-% B()%(FF,-+%B,-%B,% 9(-=,%)*+%/2B0-

13 Projective Transformations?(88,;%(%.##/")0.&%%!2/%01)+)"-0,"20,345,-)0- Image warping with homographies image plane in front image plane below black area where no pixel maps to

14 Camera Calibration Camera Model: Parameters! Camera Parameters! Intrinsic Parameters (of the camera): link the frame coordinates of an image point with its corresponding camera coordinates! Extrinsic parameters: define the location and orientation of the camera coordinate system with respect to the world coordinate system x im (x im,y im ) O y x y im p P P w Z w X w Y w

15 Camera coordinate system Principal axis: line from the camera center perpendicular to the image plane Principal point (p): point where principal axis intersects the image plane (origin of normalized coordinate system) Perspective Projection Equations (X-Z Plane) X x f Z "x f = X Z "x = f X Z Notice: Position on the image plane is related to depth. (u,v) are scaled equally based on focal length and depth

16 Equivalent Image Geometries! Consider case with object on optical axis! More convenient geometry, with an upright image! Both are equivalent mathematically x = f X Z (X,Y,Z)! ( f X Z, f Y Z ) X Y Z 1 " # % &! f X f Y Z " # % & Pinhole camera model X Y Z 1 " # % & f 0 f " # % & =

17 Principal point offset principal point: Camera coordinate system: origin is at the principal point Image coordinate system: origin is in the corner Principal point offset principal point: (X,Y,Z)! ( f X Z + p x, f Y Z + p y )

18 Pixel size Pixel size:! m x pixels per meter in horizontal direction, m y pixels per meter in vertical direction pixels/m m pixels Skew The camera coordinate system may be skewed due to some manufacturing error.

19 Lens distortion! Two main lens distortion:! Radial distortion: arise as a result of the shape of lens (e.g. fish-eye)! Tangential distortion: arise from the assembly process of the camera as a whole. Radial Distortion! Zero at the optical center of the image! Increases as we move toward the periphery! Rays farther from the center of the lens are bent more than those closer in! Terms: k1, k2, k3

20 Intrinsic parameters! Principal point coordinates! Focal length! Pixel magnification factors! Skew (non-rectangular pixels)! Radial distortion Camera parameters Extrinsic parameters! Rotation and translation relative to world coordinate system Camera rotation and translation In general, the camera coordinate frame will be related to the world coordinate frame by a rotation and a translation ( ) X cam = R X - C coords. of point in camera frame coords. of a point in world frame (nonhomogeneous) coords. of camera center in world frame

21 Intrinsic parameters! Principal point coordinates! Focal length! Pixel magnification factors! Skew (non-rectangular pixels)! Radial distortion Camera parameters Extrinsic parameters! Rotation and translation relative to world coordinate system Camera Calibration Basic matrix: " % " t 1 %" R 11 R 12 R 13 0% " T 1 % G = PLRT = t 2 R 21 R 22 R T t 3 R 31 R 32 R T 3 # 0 0 1/ f 0& # &# & # & Intrinsic Parameters Extrinsic Parameters How to estimate those parameters?

22 Camera calibration Given n points with known 3D coordinates X i and known image projections x i, estimate the camera parameters X i x i Example Calibration Pattern Harris Corner Detector From Sebastian Thrun 44

23 Camera calibration Calibration: 2 steps! Step 1: Transform into camera coordinates " X C % " X W % Y C # Z C = f ( Y W & Z W,(,),*,T) # &! Step 2: Transform into image coordinates 46

24 Calibration Model (extrinsic) 47 Advantage of Homogeneous C s i-th data point 48

25 Camera Calibration! Algorithm 1. Measure N 3D coordinates (Xi, Yi,Zi) 2. Locate their corresponding image points (xi,yi) - Edge, Corner, Hough 3. Build matrix A of a homogeneous system Av = 0 4. Compute SVD of A, solution v 5. Determine aspect ratio # and scale 6. Recover the first two rows of R and the first two components of T up to a sign 7. Determine sign s of by checking the projection equation 8. Compute the 3 rd row of R by vector product, and enforce orthogonality constraint by SVD 9. Solve Tz and fx using Least Square and SVD, then fy = fx / #% Y w Z w X w How Many Images Do We Need?! Assumption: K images with M corners each! 4+6K parameters! 2KM constraints! 2KM & 4+6K M>3 and K &2/(M-3)! 2 images with 4 points, but will 1 images with 5 points work?! No, since points cannot be co-planar! 50

26 Calibration tools? Multi-view Geometry

27 Two-view geometry Scene geometry (structure): Given corresponding points in two or more images, where is the pre-image of these points in 3D? Correspondence (stereo matching): Given a point in just one image, how does it constrain the position of the corresponding point x in another image? Camera geometry (motion): Given a set of corresponding points in two images, what are the cameras for the two views? Parameters of a Stereo System! Intrinsic Parameters! Characterize the transformation from camera to pixel coordinate systems of each camera! Focal length, image center, aspect ratio P l P P r! Extrinsic parameters! Describe the relative position and orientation of the two cameras X l! Rotation matrix R and translation vector T p l Y l f l O l Z l R, T Z r X r p r Y r f r O r Bahadir K. Gunturk

28 Triangulation Given projections of a 3D point in two or more images (with known camera matrices), find the coordinates of the point X? x 1 x 2 O 1 O 2 Triangulation We want to intersect the two visual rays corresponding to x 1 and x 2, but because of noise and numerical errors, they don t meet exactly R 2 X? R 1 x 1 x 2 O 1 O 2

29 Triangulation: Geometric approach Find shortest segment connecting the two viewing rays and let X be the midpoint of that segment X x 1 x 2 O 1 O 2 Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the other camera center = vanishing points of camera motion direction Epipolar Lines - intersections of epipolar plane with image planes (always come in corresponding pairs)

30 Epipolar constraint X x x If we observe a point x in one image, where can the corresponding point x be in the other image? Epipolar constraint X X X x x x x Potential matches for x have to lie on the corresponding epipolar line l. Potential matches for x have to lie on the corresponding epipolar line l.

31 Calibrated Camera Essential matrix Bahadir K. Gunturk Example: Converging cameras

32 Rectification! Problem: Epipolar lines not parallel to scan lines P P l P r Epipolar Plane p l Epipolar Lines p r O l e l e r Or Epipoles From Sebastian Thrun/Jana Kosecka Rectification! Problem: Epipolar lines not parallel to scan lines P Epipolar Plane P l P r Rectified Images p l Epipolar Lines p r O l Or Epipoles at infinity From Sebastian Thrun/Jana Kosecka

33 Epipolar rectification Rectified Image Pair Epipolar rectification Rectified Image Pair

34 Stereo results Scene Ground truth (from Seitz) Results with window correlation Window-based matching (best window size) Ground truth (from Seitz)

35 Results with better method State of the art Ground truth Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September (from Seitz) How can We Improve Stereo? Space-time stereo scanner uses unstructured light to aid in correspondence Result: Dense 3D mesh (noisy) From Sebastian Thrun/Jana Kosecka

36 Unstructured Light 3D Scanner OpenCV! cvfindchessboardcorners()! Calibrate:! cvcalibratecamera2()! cvstereocalibrate()! Rectify / Epipolar! cvinitundistortmap()! cvinitundistortrectifymap()! cvcomputecorrespondepilines()! cvfindfundamentalmatrix()! cvstereorectify()! Stereo! cvfindstereocorrespondencebm()! Depth maps! cvreprojectimageto3d()

37 Assignment #4! Download the Matlab Camera Calibration Toolbox! Read Documentation! First calibration example! Download data! Run camera calibration software! Submit results Conclusion! There s a mathematical model to describe cameras! Parameters can be estimated using a camera calibration approach! Camera calibration is key for accurate stereo reconstruction

Pin Hole Cameras & Warp Functions

Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Motivation Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg

Computer Vision Projective Geometry and Calibration. Pinhole cameras

Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole

3D Geometry and Camera Calibration

3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often

Camera Model and Calibration

Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the

Lecture 9: Epipolar Geometry

Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1)

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture

Lecture 14: Basic Multi-View Geometry

Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram

CV: 3D to 2D mathematics. Perspective transformation; camera calibration; stereo computation; and more

CV: 3D to 2D mathematics Perspective transformation; camera calibration; stereo computation; and more Roadmap of topics n Review perspective transformation n Camera calibration n Stereo methods n Structured

There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...

STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own

Reminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches

Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary

Camera Model and Calibration. Lecture-12

Camera Model and Calibration Lecture-12 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the

Rectification and Disparity

Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah

Camera Models and Image Formation Srikumar Ramalingam School of Computing University of Utah srikumar@cs.utah.edu Reference Most slides are adapted from the following notes: Some lecture notes on geometric

Computer Vision. Coordinates. Prof. Flávio Cardeal DECOM / CEFET- MG.

Computer Vision Coordinates Prof. Flávio Cardeal DECOM / CEFET- MG cardeal@decom.cefetmg.br Abstract This lecture discusses world coordinates and homogeneous coordinates, as well as provides an overview

Lecture 9 & 10: Stereo Vision

Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving

Rectification. Dr. Gerhard Roth

Rectification Dr. Gerhard Roth Problem Definition Given a pair of stereo images, the intrinsic parameters of each camera, and the extrinsic parameters of the system, R, and, compute the image transformation

Camera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah

Camera Models and Image Formation Srikumar Ramalingam School of Computing University of Utah srikumar@cs.utah.edu VisualFunHouse.com 3D Street Art Image courtesy: Julian Beaver (VisualFunHouse.com) 3D

Epipolar geometry. x x

Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections

Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy

1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:

Recovering structure from a single view Pinhole perspective projection

EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their

Introduction to Homogeneous coordinates

Last class we considered smooth translations and rotations of the camera coordinate system and the resulting motions of points in the image projection plane. These two transformations were expressed mathematically

Lecture 6 Stereo Systems Multi-view geometry

Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems

Midterm Exam Solutions

Midterm Exam Solutions Computer Vision (J. Košecká) October 27, 2009 HONOR SYSTEM: This examination is strictly individual. You are not allowed to talk, discuss, exchange solutions, etc., with other fellow

Projective Geometry and Camera Models

/2/ Projective Geometry and Camera Models Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Note about HW Out before next Tues Prob: covered today, Tues Prob2: covered next Thurs Prob3:

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004

Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,

Projective Geometry and Camera Models

Projective Geometry and Camera Models Computer Vision CS 43 Brown James Hays Slides from Derek Hoiem, Alexei Efros, Steve Seitz, and David Forsyth Administrative Stuff My Office hours, CIT 375 Monday and

Laser sensors. Transmitter. Receiver. Basilio Bona ROBOTICA 03CFIOR

Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The receiver measures the time of flight (back and

3D Vision Real Objects, Real Cameras. Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun,

3D Vision Real Objects, Real Cameras Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun, anders@cb.uu.se 3D Vision! Philisophy! Image formation " The pinhole camera " Projective

Stereo vision. Many slides adapted from Steve Seitz

Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is

Module 4F12: Computer Vision and Robotics Solutions to Examples Paper 2

Engineering Tripos Part IIB FOURTH YEAR Module 4F2: Computer Vision and Robotics Solutions to Examples Paper 2. Perspective projection and vanishing points (a) Consider a line in 3D space, defined in camera-centered

Models and The Viewing Pipeline. Jian Huang CS456

Models and The Viewing Pipeline Jian Huang CS456 Vertex coordinates list, polygon table and (maybe) edge table Auxiliary: Per vertex normal Neighborhood information, arranged with regard to vertices and

3D FACE RECONSTRUCTION BASED ON EPIPOLAR GEOMETRY

IJDW Volume 4 Number January-June 202 pp. 45-50 3D FACE RECONSRUCION BASED ON EPIPOLAR GEOMERY aher Khadhraoui, Faouzi Benzarti 2 and Hamid Amiri 3,2,3 Signal, Image Processing and Patterns Recognition

Perspective Projection [2 pts]

Instructions: CSE252a Computer Vision Assignment 1 Instructor: Ben Ochoa Due: Thursday, October 23, 11:59 PM Submit your assignment electronically by email to iskwak+252a@cs.ucsd.edu with the subject line

Multi-view geometry problems

Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera

3D Viewing. CS 4620 Lecture 8

3D Viewing CS 46 Lecture 8 13 Steve Marschner 1 Viewing, backward and forward So far have used the backward approach to viewing start from pixel ask what part of scene projects to pixel explicitly construct

CS 231A: Computer Vision (Winter 2018) Problem Set 2

CS 231A: Computer Vision (Winter 2018) Problem Set 2 Due Date: Feb 09 2018, 11:59pm Note: In this PS, using python2 is recommended, as the data files are dumped with python2. Using python3 might cause

Lecture 3: Camera Calibration, DLT, SVD

Computer Vision Lecture 3 23--28 Lecture 3: Camera Calibration, DL, SVD he Inner Parameters In this section we will introduce the inner parameters of the cameras Recall from the camera equations λx = P

Image Formation I Chapter 2 (R. Szelisky)

Image Formation I Chapter 2 (R. Selisky) Guido Gerig CS 632 Spring 22 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified from

Visual Recognition: Image Formation

Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION

CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes

Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision

Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Professor Willia Hoff Dept of Electrical Engineering &Coputer Science http://inside.ines.edu/~whoff/ 1 Caera Calibration 2 Caera Calibration Needed for ost achine vision and photograetry tasks (object

Lecture 7 Measurement Using a Single Camera. Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016

Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016 If I have an image containing a coin, can you tell me the diameter of that coin? Contents

Compositing a bird's eye view mosaic

Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching

Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?

Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple

Final Exam Study Guide CSE/EE 486 Fall 2007

Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.

Structure from Motion. Prof. Marco Marcon

Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)

Agenda. Perspective projection. Rotations. Camera models

Image formation Agenda Perspective projection Rotations Camera models Light as a wave + particle Light as a wave (ignore for now) Refraction Diffraction Image formation Digital Image Film Human eye Pixel

Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras

Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/ Outline The geometry of active stereo.

Scalable geometric calibration for multi-view camera arrays

Scalable geometric calibration for multi-view camera arrays Bernhard Blaschitz, Doris Antensteiner, Svorad Štolc Bernhard.Blaschitz@ait.ac.at AIT Austrian Institute of Technology GmbH Intelligent Vision

Basilio Bona DAUIN Politecnico di Torino

ROBOTICA 03CFIOR DAUIN Politecnico di Torino Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The

Single View Geometry. Camera model & Orientation + Position estimation. What am I?

Single View Geometry Camera model & Orientation + Position estimation What am I? Vanishing points & line http://www.wetcanvas.com/ http://pennpaint.blogspot.com/ http://www.joshuanava.biz/perspective/in-other-words-the-observer-simply-points-in-thesame-direction-as-the-lines-in-order-to-find-their-vanishing-point.html

Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15

Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems

Geometry of image formation

eometry of image formation Tomáš Svoboda, svoboda@cmp.felk.cvut.cz Czech Technical University in Prague, Center for Machine Perception http://cmp.felk.cvut.cz Last update: November 3, 2008 Talk Outline

CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching

Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix

Projector Calibration for Pattern Projection Systems

Projector Calibration for Pattern Projection Systems I. Din *1, H. Anwar 2, I. Syed 1, H. Zafar 3, L. Hasan 3 1 Department of Electronics Engineering, Incheon National University, Incheon, South Korea.

Depth from two cameras: stereopsis

Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture

Stereo Vision Based Mapping and Immediate Virtual Walkthroughs

Stereo Vision Based Mapping and Immediate Virtual Walkthroughs Heiko Hirschmüller M.Sc., Dipl.-Inform. (FH) Submitted in partial fulfilment of the requirements for the degree of DOCTOR OF PHILOSOPHY June

Camera Calibration using Vanishing Points

Camera Calibration using Vanishing Points Paul Beardsley and David Murray * Department of Engineering Science, University of Oxford, Oxford 0X1 3PJ, UK Abstract This paper describes a methodformeasuringthe

Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds

Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds Marta Wilczkowiak Edmond Boyer Peter Sturm Movi Gravir Inria Rhône-Alpes, 655 Avenue de l Europe, 3833 Montbonnot, France

Image warping and stitching

Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

Three-Dimensional Viewing Hearn & Baker Chapter 7

Three-Dimensional Viewing Hearn & Baker Chapter 7 Overview 3D viewing involves some tasks that are not present in 2D viewing: Projection, Visibility checks, Lighting effects, etc. Overview First, set up

Image Warping and Mosacing

Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

Binocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?

Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo

Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the

Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the corresponding 3D points. The projection models include:

1 CSE 252A Computer Vision I Fall 2017

Assignment 1 CSE A Computer Vision I Fall 01 1.1 Assignment This assignment contains theoretical and programming exercises. If you plan to submit hand written answers for theoretical exercises, please

CSE 167: Introduction to Computer Graphics Lecture #3: Coordinate Systems

CSE 167: Introduction to Computer Graphics Lecture #3: Coordinate Systems Jürgen P. Schulze, Ph.D. University of California, San Diego Spring Quarter 2015 Announcements Project 2 due Friday at 1pm Homework

Introduction to Computer Vision. Introduction CMPSCI 591A/691A CMPSCI 570/670. Image Formation

Introduction CMPSCI 591A/691A CMPSCI 570/670 Image Formation Lecture Outline Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic

Jump Stitch Metadata & Depth Maps Version 1.1

Jump Stitch Metadata & Depth Maps Version 1.1 jump-help@google.com Contents 1. Introduction 1 2. Stitch Metadata File Format 2 3. Coverage Near the Poles 4 4. Coordinate Systems 6 5. Camera Model 6 6.

Image warping and stitching

Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

Transforms. COMP 575/770 Spring 2013

Transforms COMP 575/770 Spring 2013 Transforming Geometry Given any set of points S Could be a 2D shape, a 3D object A transform is a function T that modifies all points in S: T S S T v v S Different transforms

Single-view metrology

Single-view metrology Magritte, Personal Values, 952 Many slides from S. Seitz, D. Hoiem Camera calibration revisited What if world coordinates of reference 3D points are not known? We can use scene features

Overview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers

Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented

A Stratified Approach for Camera Calibration Using Spheres

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, MONTH YEAR 1 A Stratified Approach for Camera Calibration Using Spheres Kwan-Yee K. Wong, Member, IEEE, Guoqiang Zhang, Student-Member, IEEE and Zhihu

Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Stereo Vision 2 Inferring 3D from 2D Model based pose estimation single (calibrated) camera Stereo

Vision 3D articielle Multiple view geometry

Vision 3D articielle Multiple view geometry Pascal Monasse monasse@imagine.enpc.fr IMAGINE, École des Ponts ParisTech Contents Multi-view constraints Multi-view calibration Incremental calibration Global

Rectification for Any Epipolar Geometry

Rectification for Any Epipolar Geometry Daniel Oram Advanced Interfaces Group Department of Computer Science University of Manchester Mancester, M13, UK oramd@cs.man.ac.uk Abstract This paper proposes

Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC

Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC Shuji Sakai, Koichi Ito, Takafumi Aoki Graduate School of Information Sciences, Tohoku University, Sendai, 980 8579, Japan Email: sakai@aoki.ecei.tohoku.ac.jp

Part I: Single and Two View Geometry Internal camera parameters

!! 43 1!???? Imaging eometry Multiple View eometry Perspective projection Richard Hartley Andrew isserman O p y VPR June 1999 where image plane This can be written as a linear mapping between homogeneous

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller

3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza

Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,

Computer Vision. Geometric Camera Calibration. Samer M Abdallah, PhD

Computer Vision Samer M Abdallah, PhD Faculty of Engineering and Architecture American University of Beirut Beirut, Lebanon Geometric Camera Calibration September 2, 2004 1 Computer Vision Geometric Camera

Image warping and stitching

Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.

CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 4 Affine Structure from Motion Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D, Newmark Civil Engineering

Exterior Orientation Parameters

Exterior Orientation Parameters PERS 12/2001 pp 1321-1332 Karsten Jacobsen, Institute for Photogrammetry and GeoInformation, University of Hannover, Germany The georeference of any photogrammetric product

SRI Small Vision System

Small Vision System Calibration 1 SRI Small Vision System Calibration Supplement to the User s Manual Software version 2.2b July 2001 Kurt Konolige and David Beymer SRI International konolige@ai.sri.com

Stereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision

Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras

Omni Stereo Vision of Cooperative Mobile Robots

Omni Stereo Vision of Cooperative Mobile Robots Zhigang Zhu*, Jizhong Xiao** *Department of Computer Science **Department of Electrical Engineering The City College of the City University of New York (CUNY)

Lecture 5.3 Camera calibration. Thomas Opsahl

Lecture 5.3 Camera calibration Thomas Opsahl Introduction The image u u x X W v C z C World frame x C y C z C = 1 For finite projective cameras, the correspondence between points in the world and points

Mapping Non-Destructive Testing Data on the 3D Geometry of Objects with Complex Shapes

More Info at Open Access Database www.ndt.net/?id=17670 Mapping Non-Destructive Testing Data on the 3D Geometry of Objects with Complex Shapes Abstract by S. Soldan*, D. Ouellet**, P.Hedayati**, A. Bendada**,

Perspective Correction Methods for Camera-Based Document Analysis

Perspective Correction Methods for Camera-Based Document Analysis L. Jagannathan and C. V. Jawahar Center for Visual Information Technology International Institute of Information Technology Gachibowli,

Robust Geometry Estimation from two Images

Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based

CSE 252B: Computer Vision II

CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points