Image Processing: 3D Image Warping. Spatial Transformations. Spatial Transformations. Affine Transformations. Affine Transformations CS334

Transcription

1 Image Warping Image rocessing: 3D Image Warping We can divide image arping into Simple spatial transformations (no per-pixel depth information) Full 3D transformations (needs per-pixel depth information) (sometimes called 3D Image Warping ) CS334 Spatial Transformations Spatial Transformations A geometric relationship beteen input (u,v) and output pixels (x,y) Forard mapping: (x,y) = (X(u,v), Y(u,v)) Inverse mapping: (u,v) = (U(x,y), V(x,y)) Matrix form is [x, y, ] = [u, v, ] T here T = a a 2 a 3 a 2 a 22 a 23 a 3 a 32 a 33 and operates in the homogeneous coordinate system. (examples from Wolberg) Affine Transformations Affine Transformations Matrix form is [x, y, ] = [u, v, ] T The T matrix can be inferred from correspondences here T = a a 2 0 a 2 a 22 0 a 3 a 32 and accommodates translations, rotations, scale, and shear. (examples from Wolberg)

2 Affine Transformations The T matrix can be inferred from correspondences x 0 y 0 u 0 v 0 a a 2 0 x y u v a 2 a 22 0 x 2 y 2 u 2 v 2 a 3 a 32 Given 3 correspondences solve for the T matrix erspective Transformations Matrix form is [x, y, ] = [u, v, ] T here T = a a 2 a 3 a 2 a 22 a 23 a 3 a 32 a 33 and in addition accommodates foreshortening of distant line and convergence of lines to a vanishing point (only parallel lines parallel to the projection plane remain parallel). erspective Transformations The T matrix can be inferred from correspondences erspective Transformations The T matrix can be inferred from correspondences u 0 v u 0 v 0 v 0 x 0 u v u x v x u 2 v u 2 x 2 v 2 x 2 u 3 v u 3 x 3 v 3 x u 0 v 0 -u 0 y 0 v 0 y u v -u y v y u 2 v 2 -u 2 y 2 v 2 y u 3 v 3 -u 3 y 3 v 3 y 3 A = X here A is the vector of unknon coefficients a ij 3D Image Warping Introduction What s it good for? Equations Ho the heck do you do it? Examples What does it look like? Misc. Issues Disocclusions Rendering order Splatting 3D Image Warping Goal: arp the pixels of the image so that they appear in the correct place for a ne viepoint Advantage: Don t need a geometric model of the object/environment Can be done in time proportional to screen size and (mostly) independent of object/environment complexity Disadvantage: Limited resolution Excessive arping reveals several visual artifacts (see examples) 2

3 Some pictures courtesy of SIGGRAH 99 course notes (Leonard McMillan) McMillan & Bishop Warping Equation: x 2 = δ(x ) 2 - (c -c 2 ) x Images enhanced ith per-pixel depth [McMillan95] Move pixels based on distance to eye ~Texture mapping u er-pixel distance values are used to arp pixels to their correct location for the current eye position v u = C + ( c + ua + vb ) C = C u = C+ ( c = C 2 + ( c2 u 2 + u a + v b) + u a2 + v b2) v / also called generalized disparity v 2 v 2 another notation δ(u, v ) C C C 2 3

4 3D Image u2 = 3 v2 = u u 3 33 u u v 4 δ ( u, v) v δ ( u, v ) v v δ ( u, v ) δ u, v ) ( u u 2 v v 2 C C 2 3D Image 3D Image DeltaSphere Lars Nyland et al. 3D Image 3D Image 4

5 3D Image Disocclusions Disocclusions (or exposure events) occur hen unsampled surfaces become visible What can e do? Disocclusions Rendering Order Bilinear patches: fill in the areas What else? Occlusion Compatible Rendering Order Occlusion Compatible Rendering Order roject the ne viepoint onto the original image and divide the image into, 2 or 4 sheets eye Reference CO A raster scan of each sheet produces a back-tofront ordering of arped pixels 5

6 Splatting More Examples Using the DeltaSphere One pixel in the source image does not necessarily project to one pixel in the destination image e.g., if you are alking toards something, the sample might get larger Lars Nyland et al. A solution: estimate shape and size of footprint of arped samples expensive to do accurately square/rectangular approximations can be done quickly (3x3 or 5x5 splats) occlusion-compatible rendering ill take care of oversized splats BUT large splats can make the image seem blocky/lo-res courtesy 3 rd Tech Inc. 300 o x 300 o panorama this is the reflected light 300 o x 300 o panorama this is the range light spherical range panoramas Jeep one scan planar re-projection 6

7 Jeep one scan Complete Jeep model 7