Image-Bas ed R endering Using Image Warping. Conventional 3-D Graphics

Transcription

1 Image-Bas ed R endering Using Image Warping Leonard McMillan LCS Computer Graphics Group MIT Conventional 3-D Graphics Simulation

2 Computer Vis ion Analys is T he Image-Bas ed Approach T rans formation

3 Images as a Collection of R ays An image is a subset of the rays seen from a given point - this s pace of rays occupies two dimens ions T he Plenoptic F unction T he set of rays seen from all points... p = P( θ, φ, x, y, z, λ, t) 3

4 Image-based rendering is about recons tructing a plenoptic function from a set of samples taken from it. Ignoring time, and s electing a dis crete set of wavelengths gives a 5-D plenoptic function Where to Begin? Pinhole camera model z Defines a mapping from image points to rays in space 4

5 5 Mapping from Rays to Points Mapping from Rays to Points Simple Derivation Corres pondence Corres pondence ( ) ( ), ( ) P x P C C P x P x P C C P x P x P t C C P x t P x t C C P x t P x t C P x t C H e t t t t δ + = + = + = + = + = +

6 Planar Warping E quation x ( x) P ( C C ) + P P = δ x. C 3 X... δ(x )(C -C 3 ) P x r. C P x P x.. δ(x )(C -C ) P x. C Resulting Warping Function A perturbed planar warp... x = δe + H x 6

7 S pecial Case A s imple P lanar warp x = H x A 3D Warp Warping in Action 7

8 Vis ibility T he warping equation determines where points go but that is not s ufficient Partition Reference Image Project the des ired center-of-projection onto the reference image 8

9 Enumeration Drawing toward the projected point guarantees an occlusion compatible ordering Ordering is cons is tent with a painter's algorithm Independent of the s cene's contents Easily generalized to other viewing s urfaces No auxiliary information required R econs truction T ypical images are discrete, not continuous An image can be formed by different geometries 9

10 Gaus s ian Cloud Model Represents samples as Gaus s ian cloud dens ities Excessive exposure errors Bilinear Patch Model Fits a bilinear patch through grid points in reference image E xces s ive occlus ion errors 0

11 Comparis on of Models Gaussian-Cloud Model Bilinear-Patch Model z E xces s ive expos ure errors z Pinhole problems z Generally preferred z E xces sive occlusion errors z Rasterization H/W z Difficult to navigate Problems with Planar Cameras Invis ible occluder problem 5 intrinsic parameters Non-uniform sampling of solid angle

12 Panoramic Cameras Warping equation can be easily adapted Visibility algorithm works Nonlinear mapping functions Examples Cylindrical camera

13 Cons tructing Panoramas Images are related by a projective trans forms x = H x Optimization problem z maximize normalized correlation z minimize s um of s quared error Initial Guesses and Constraints S um of angles is π z cons trains focal length S kew of camera is near 0 As pect ratio near 3

14 Finding Disparity How to get it z 3-D laser scanners z Depth-from-stereo z Depth-from-motion z Depth-from-focus z Depth-from-light-fields z Manual layer s egmentation How accurate must it be? Vis ual Hulls Depth-from-s ilhouettes S imple computer vis ion methods z blue s creening z image differencing Can be computed in image space 4

15 Image-bas ed Vis ual Hulls Volume-like Self-consistent Dis cretedis cretecontinuous Depth from R edundant S tructure Light fields for depth acquisition z Depth-from -stereo -motion -focus - silhouettes 5

16 Comparing Rendering Approaches Geometry B as ed z Forward Mapping (graphics pipeline) z Invers e Mapping (ray tracing) Image based z Greater spatial coherence z Lower depth complexity Image-Based Pipeline 6

17 Forward-warping S ingle depth value per pixel F orward-mapped Vis ual Hull Draws a line segment for each interval 7

18 Merging Forward Warps Draw textured line s egments Image-Bas ed R ay T racing Inverse warping 8

19 Algorithm Properties S earch is confined to a line F irst inters ection is clos es t point Incremental line drawing R econs truction in reference image Work is proportional to s ize of output image Applications of IB R IBR combined with traditional methods Decouples rendering from interaction L atency compens ation 9

20 Conclus ions IBR provides z new representations for 3D graphics - eas y to acquire - allows efficient rendering z scalable performance - depends on number of pixels rather than the number of geometric primitives z amenable to HW acceleration 0