Image Processing: Motivation Rendering from Images. Related Work. Overview. Image Morphing Examples. Overview. View and Image Morphing CS334

Transcription

1 Motivation Rendering from Images Image rocessing: View and CS334 Given left image right image Create intermediate images simulates camera movement [Seitz96] Related Work anoramas ([Chen95/QuicktimeVR], etc) user can look in any direction at few given locations but camera translations are not allowed Overview View morphing image pre-warping image morphing image post-warping Quicktime VR Demo Overview Examples View morphing image pre-warping image morphing image post-warping 1

2 Identify correspondences between input/output image roduce a sequence of images that allow a smooth transition from the input image to the output image 0 k n 2. Linear interpolation k n k n k = (1 ) 0 + n frame 0 frame k frame n 2

3 is not shape preserving Overview View Morphing Examples View morphing image pre-warping image morphing image post-warping View Morphing Shape preserving morph Three step algorithm rewarp first and last images to parallel views Image morph between prewarped images ostwarp to interpolated view arallel views same image plane image plane parallel to segment connecting the two centers of projection 0 n rewarp np compute parallel views, p rotate and to parallel views prewarp correspondence is ( 0, n ) -> ( op, np ) p 3

4 Step 3: ostwarping 0 n Shape preserving Use prewarped correspondences Interpolate from ostwarp morphed image create intermediate view is known interpolate view direction and tilt rotate morphed image to intermediate view kp np p View morphing View morphing View morphing is shape preserving View Morphing More Examples Using computer vision/stereo reconstruction techniques Overview View morphing, more details for synthetic rendering image pre-warping image morphing image post-warping 4

5 arallel views arallel views use for x (a p vector) 0 n 0 n np np p p arallel views use for x (a p vector) use (a 0 x b 0 ) x (a n x b n ) as y (-b p ) arallel views use for x (a p vector) use (a 0 x b 0 ) x (a n x b n ) as y (-b p ) use z = x cross y 0 n 0 n np np p p arallel views use for x (a p vector) use (a 0 x b 0 ) x (a n x b n ) as y (-b p ) use z = x cross y 0 use same pixel size n use wider field of view 0 n prewarping using reprojection of rays np np p p 5

6 0 n prewarping using reprojection of rays look up all the rays of the prewarped view in the original view prewarping using reprojection of rays look up all the rays of the prewarped view in the original view 0 alternative: prewarping using n texture mapping np np p p prewarping using reprojection of rays look up all the rays of the prewarped view in the original view 0 n np alternative: prewarping using texture mapping create polygon for image plane consider it texture mapped with the image itself render scene from prewarped view I if you go this path you will have to np implement clipping with prewarped plane note: texture mapping in hardware 0 prewarping correspondences for all pairs of correspondence 0 n n np p 0 n prewarping correspondences for all pairs of correspondence 0 n project 0 on, computing project n on p, computing np prewarped correspondence is op np 0 n use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to intermediate kp np np p p 6

7 use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to 0 intermediate n use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to 0 intermediate n kp np p corresponding pixels are on same line in prewarped views kp np corresponding pixels are on same line in prewarped views preventing holes p use prewarped correspondences to compute a correspondence for all pixels in I I 0p 0 linearly interpolate to intermediate 0 kp n corresponding pixels are on same line in prewarped views preventing holes or linearly interpolate between np I consecutive samples np or postprocess morphed image looking for background pixels and C replacing them with neighboring k values use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to intermediate corresponding pixels are on same line in prewarped views preventing holes or linearly interpolate between consecutive samples or postprocess morphed image looking p for background pixels and replacing them with neighboring values visibility artifacts collision of samples 0 in prewarped views kp n np use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to intermediate corresponding pixels are on same line in prewarped views preventing holes or linearly interpolate between consecutive samples or postprocess morphed image looking p for background pixels and replacing them with neighboring values visibility artifacts collision of samples Zbuffer of disparity 0 in prewarped views kp n np use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to intermediate corresponding pixels are on same line in prewarped views ti h l or linearly interpolate between consecutive samples or postprocess morphed image looking for background pixels and replacing p them with neighboring values visibility artifacts collision of samples Zbuffer of disparity holes 0 preventing holes kp n np 7

8 Step 3: ostwarping use prewarped correspondences to compute a correspondence for all pixels in linearly interpolate to intermediate corresponding pixels are on same line in prewarped views ti h l or linearly interpolate between consecutive samples or postprocess morphed image looking for background pixels and replacing p them with neighboring values visibility artifacts collision of samples Zbuffer of disparity holes morph p to I kp use additional views 0 preventing holes kp n np create intermediate view is known current view direction is a linear interpolation of the start and end view directions current up vector is a linear interpolation of the start and end up vectors rotate morphed image to intermediate view same as prewarping 8