Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps
Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions.
Motivation
Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg Pinhole Camera
Pinhole Camera Real camera image is inverted Instead model impossible but more convenient virtual image
Pinhole Camera Terminology
Normalized Camera By similar triangles:
Focal length parameters
Focal length parameters Can model both the effect of the distance to the focal plane the density of the receptors with a single focal length parameter φ In practice, the receptors may not be square: So use different focal length parameter for x and y dims
Offset parameters Current model assumes that pixel (0,0) is where the principal ray strikes the image plane (i.e. the center) Model offset to center
Skew parameter Finally, add skew parameter Accounts for image plane being not exactly perpendicular to the principal ray
Radial distortion
Camera & World Coordinates w w 0 u 0 camera coordinate frame o 0 o world coordinate frame u apple apple apple apple u 0!1! 2 u x w + w 0 = Rotation Matrix Translation Vector! 3! 4 z
Position and orientation of camera Position w=(u,v,w) T of point in the world is generally not expressed in the frame of reference of the camera. Transform using 3D transformation or Point in frame of reference of camera Point in frame of reference of world
Constraints on As is a rotation matrix it is constrained by the following, T = I det( ) =1 We refer to these matrices as belonging to the Special Orthogonal Group - SO(3). How many degrees of freedom do you think has?
Something to try In MATLAB type, >> R1 = orth(randn(3,3)); >> R1(:,end) = det(r1)*r1(:,end); >> R2 = orth(randn(3,3)); >> R2(:,end) = det(r2)*r2(:,end); If you form a new matrix as a linear combination of R1 & R2, >> R3 = 0.5*R1 + 0.5*R2; Does R3 lie in SO(3)?
Reminder: Convex Set 17
Reminder: Non-Convex Set 18
Complete pinhole camera model Intrinsic parameters (stored as intrinsic matrix) Extrinsic parameters
Complete pinhole camera model For short: Question: is a linear function?
Perspective Transform
Learning extrinsic parameters ˆ, ˆ =min, NX n=1 {x n pinhole[w n,,, ]} e.g. {x} = x 2 2
Learning intrinsic parameters ˆ =min [min, NX n=1 {x n pinhole[w n,,, ]}] e.g. {x} = x 2 2
Camera Calibration Use 3D target with known 3D points.
For you to try.. There exists camera calibration tools in MATLAB, see Bouget s Calibration Toolbox in MATLAB. Or if you prefer, you can use OpenCV s tutorial. What are the intrinsics of your device? How sensitive are vision algorithms to the correct intrinsics?
Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions.
Homogeneous Coordinates Convert 2D coordinate to 3D To convert back
Geometric interpretation
Pinhole camera Camera model: In homogeneous coordinates: (linear!)
Pinhole camera Writing out these three equations Eliminate λ to retrieve original equations
Adding in extrinsics Or for short: Or even shorter:
Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions.
Planar Warp Functions Consider viewing a planar scene There is now a 1 to 1 mapping between points on the plane and points in the image We will investigate models for this 1 to 1 mapping Euclidean Similarity Affine Homography
Piecewise planarity Many scenes are not planar, but are nonetheless piecewise planar Can we match all of the planes to one another?
Euclidean warp Consider viewing a fronto-parallel plane at a fixed distance D. In homogeneous coordinates, the imaging equations are: 3D rotation matrix becomes 2D (in plane) Plane at known distance D Point is on plane (w=0)
Euclidean warp Simplifying Rearranging the last equation
Euclidean warp Homogeneous: Cartesian: For short: How many unknowns?
More to read Prince et al. Chapter 14, Section 1 & 3. Chapter 15, Section 1.