3D Sensing Camera Model: Recall there are 5 Different Frames of Reference c Camera Model and 3D Transformations Camera Calibration (Tsai s Method) Depth from General Stereo (overview) Pose Estimation from 2D Images (skip) 3D Reconstruction Object orld Camera Real Image Piel Image C f image a c f zc p zp pramid A object p 2 Rigid Bod Transformations in 3D Translation and Scaling in 3D zp pramid model in its own model space p p rotate translate scale instance of the object in the world 3 4 Rotation in 3D is about an ais z Rotation about Arbitrar Ais T R R2 P P θ rotation b angle θ about the ais 0 0 0 0 cos θ -sin θ 0 0 sin θ cos θ 0 0 0 0 P P One translation and two rotations to line it up with a major ais. Now rotate it about that ais. Then appl the reverse transformations (R2, R, T) to move it back. P P r r2 r3 t r2 r22 r23 t r3 r32 r33 tz 0 0 0 P P 5 6
The Camera Model How do we get an image IP from a world P? The camera model handles the rigid bod transformation from world coordinates to camera coordinates plus the perspective transformation to image coordinates. s Ipr s Ipc s image c c2 c3 c4 c2 c22 c23 c24 c3 c32 c33 camera matri C hat s in C? P P world 7. CP T R P 2. IP π(f) CP s Ip s Ip s image 0 0 0 0 0 0 0 0 /f perspective transformation CP CP C 3D in camera coordinates 8 Camera Calibration In order work in 3D, we need to know the parameters of the particular camera setup. Solving for the camera parameters is called calibration. Intrinsic Parameters principal (u0,v0) C scale factors (d,d) f c C zc c intrinsic parameters are of the camera device etrinsic parameters are where the camera sits in the world aspect ratio distortion factor γ focal length f lens distortion factor κ (models radial lens distortion) (u0,v0) 9 0 Etrinsic Parameters Calibration Object translation parameters t [t t tz] rotation matri R r r2 r3 0 r2 r22 r23 0 r3 r32 r33 0 0 0 0 Are there reall nine parameters? The idea is to snap images at different depths and get a lot of 2D-3D correspondences. 2 2
The Tsai Procedure The Tsai procedure was developed b Roger Tsai at IBM Research and is most widel used. Several images are taken of the calibration object ielding correspondences at different distances. Tsai s algorithm requires n > 5 correspondences {(i, i, zi), (ui, vi)) i,,n} between (real) image s and 3D s. In this* version of Tsai s algorithm, The real-valued (u,v) are computed from their piel positions (r,c): where u γ d (c-u0) v -d (r - v0) - (u0,v0) is the center of the image - d and d are the center-to-center (real) distances between piels and come from the camera s specs - γ is a scale factor learned from previous trials 3 * This version is for single-plane calibration. 4 Tsai s Geometric Setup camera Oc Tsai s Procedure principal p0 (0,0,zi) z image plane pi (ui,vi) Pi (i,i,zi) 3D. Given the n correspondences ((i,i,zi), (ui,vi)) Compute matri A with rows ai ai (vi*i, vi*i, -ui*i, -ui*vi, vi) These are known quantities which will be used to solve for intermediate values, which will then be used to solve for the parameters sought. 5 6 Intermediate Unknowns 2. The vector of unknowns is µ (µ, µ2, µ3, µ4, µ5): µr/t µ2r2/t µ3r2/t µ4r22/t µ5t/t where the r s and t s are unknown rotation and translation parameters. Use µ to solve for t, t, and 4 rotation parameters 2 2 2 2 2 5. Let U µ + µ2 + µ3 + µ4. Use U to calculate t. (see tet) 2/2 6. Tr the positive square root t (t ) and use it to compute translation and rotation parameters. 3. Let vector b (u,u2,,un) contain the u image coordinates. 4. Solve the sstem of linear equations A µ b r µ t r2 µ2 t r2 µ3 t r22 µ4 t t µ5 t Now we know 2 translation parameters and 4 rotation parameters. for unknown parameter vector µ. 7 ecept 8 3
Determine true sign of t and compute remaining rotation parameters. Solve another linear sstem. 7. Select an object P whose image coordinates (u,v) are far from the image center. 0. e have t and t and the 9 rotation parameters. Net step is to find tz and f. 8. Use P s coordinates and the translation and rotation parameters so far to estimate the image that corresponds to P. Form a matri A whose rows are: ai (r2*i + r22*i + t, vi) If its coordinates have the same signs as (u,v), then keep t, else negate it. and a vector b whose rows are: bi (r3*i + r32*i) * vi 9. Use the first 4 rotation parameters to calculate the remaining 5.. Solve A *v b for v (f, tz). 9 20 Almost there e use them for general stereo. 2. If f is negative, change signs (see tet). 3. Compute the lens distortion factor κ and improve the estimates for f and tz b solving a nonlinear sstem of equations b a nonlinear regression. 4. All parameters have been computed. Use them in 3D data acquisition sstems. P(r,c) C e P 2 e2 P2(r2,c2) 2 C2 2 22 For a correspondence (r,c) in image to (r2,c2) in image 2:. Both cameras were calibrated. Both camera matrices are then known. From the two camera equations we get Solve b computing the closest approach of the two skew ras. P 4 linear equations in 3 unknowns. r (b - b3*r) + (b2 - b32*r) + (b3-b33*r)z c (b2 - b3*c) + (b22 - b32*c) + (b23-b33*c)z r2 (c - c3*r2) + (c2 - c32*r2) + (c3 - c33*r2)z c2 (c2 - c3*c2) + (c22 - c32*c2) + (c23 - c33*c2)z Direct solution uses 3 equations, won t give reliable results. 23 Q solve for shortest Instead, we solve for the shortest line segment connecting the two ras and let P be its mid. V P If the ras intersected perfectl in 3D, the intersection would be P. 24 4
Application: Kari Pulli s Reconstruction of 3D Objects from light-striping stereo. Application: Zhenrong Qian s 3D Blood Vessel Reconstruction from Visible Human Data 25 26 5