Mysteries of Parameterizing Camera Motion - Part 1 Instructor - Simon Lucey 16-623 - Advanced Computer Vision Apps
Today Motivation SO(3) Convex? Exponential Maps SL(3) Group.
Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince Motivation
Motivation Taken from: H. Liu et al. Robust Keyframe-based Monocular SLAM for Augmented Reality, ISMAR 2016.
Motivation The cathedral dataset: 480 camera matrices [ i, i ] Total dof = 480 (3 + 3) = 2880 91178 3D points. Total dof = 91178 3 = 273543 Adapted from: Optimization Methods in Computer Vision. Anders Eriksson
Structure from Motion FX NX x f n (w n ; f ) 2 2 f=1 n=1 x 2D projection w 3D point extrinsics N no. of points F projection function no. of frames
Reminder: Cheat Sheet Description 3D Point 2D Point Rotation matrix Intrinsics matrix Hartley & Zisserman X x R K Prince w x Homography matrix H translation vector t
Reminder: Extrinsics w w 0 u 0 camera coordinate frame o 0 o world coordinate frame u apple apple apple apple u 0!1! 2 u x w + w 0 = Rotation Matrix Translation Vector! 3! 4 z
Reminder: Extrinsics Position w=(u,v,w) T of point in the world is generally not expressed in the frame of reference of the camera. Transform using 3D transformation or Point in frame of reference of camera Point in frame of reference of world
Structure from Motion FX NX arg min W, x f n (w n ; f ) 2 2 f=1 n=1 apple w 1 (w; ) =A( )
Weak Perspective Weak perspective is an orthographic projection plus scaling. Approximates perspective when, we assume all points on a 3D object are roughly the same distance from the camera. the size of the model in depth is small compared to the depth of the model centroid d c >> d m Computations much simpler than full perspective. d c depth of model centroid from camera d m depth of model
Structure from Motion FX NX arg min W, x f n (w n ; f ) 2 2 f=1 n=1 apple w 1 (w; ) =A( ) A( ) =s apple 1 0 0 0 1 0 [, ]
X A W 3 N 2F N Is this problem convex? Known Unknown Tomasi & Kanade. 1992 13
Today Motivation SO(3) Convex? Exponential Maps SL(3) Group.
Convex Optimization A convex function x 0 and x 1 in S D f : S D! R is one that satisfies, for any f([1 ]x 0 + x 1 ) apple [1 ]f(x 0 )+ f(x 1 ) s.t. 0 apple apple 1 f(x 0 ) f(x 1 ) x 0 x 1
Convex Objectives Common convex objective functions in computer vision, f(x) = x 2 2! L 2 penalty f(x) = x 1! L 1 penalty (Quadratic Programming Problem) (Linear Programming Problem) L 2 f(x) L 1 x
Quadratic Programming Most widely used in vision and learning. arg min x x T Px + q T x + r s.t. Gx apple h Ax = b x 2 S D Examples - SfM, Support Vector Machines, Alignment, etc.
Properties Two important properties:- 1. A convex function has a single minimum on a convex domain. S S
Properties Two important properties:- 1. A convex function has a single minimum on a convex domain. S S S is not convex!!!
Example - SO(3) is not a Convex Domain As is a rotation matrix it is constrained by the following, T = I det( ) =1 We refer to these matrices as belonging to the Special Orthogonal Group - SO(3). 1, 2 2 SO(3) 1 +(1 ) 2 2/ SO(3), 8 s.t. 0 apple apple 1
Review - Something to try In MATLAB type, >> R1 = orth(randn(3,3)); >> R1(:,end) = det(r1)*r1(:,end); >> R2 = orth(randn(3,3)); >> R2(:,end) = det(r2)*r2(:,end); If you form a new matrix as a linear combination of R1 & R2, >> R3 = 0.5*R1 + 0.5*R2; Does R3 lie in SO(3)?
Today Motivation SO(3) Convex? Exponential Maps SL(3) Group.
Review: Pinhole Camera Real camera image is inverted Instead model impossible but more convenient virtual image Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince
Structure from Motion FX NX arg min W, x f n (w n ; f ) 2 2 f=1 n=1 apple w 1 (w; ) =A( )
Gauss-Newton Algorithm Gauss-Newton algorithm common strategy for optimizing non-linear least-squares problems. arg min x y F(x) 2 2 s.t. F : R N! R M Step 1: arg min x y F(x) @F(x) @x T x 2 2 Carl Friedrich Gauss Step 2: x x + x keep applying steps until x converges. Isaac Newton 26
Initialization Initialization has to be suitably close to the ground-truth for method to work. Kind of like a black hole s event horizon. You got to be inside it to be sucked in!
Gauss-Newton Optimization Can we approximate the true objective with a convex one?
Reminder: Convex Set 29
Reminder: Non-Convex Set 30
Euler Angles We could consider Euler Angles right???? R( x, y, z )=R( x )R( y )R( z ) Euler angles do form a convex set.. 0 apple apple 2 Do Euler Angles form a unique rotation? 31
Gimbal Lock 32