Rendering Synthetic Objects into Real Scenes. based on [Debevec98]

Transcription

1 Rendering Synthetic Objects into Real Scenes based on [Debevec98]

2 Compositing of synthetic objects Geometry consistency needed: geometric model of synthetic objects needed: (coarse) geometric model of the real scene needed: camera calibration (extrinsics & intrinsics) procedure: rendering of synthetic and real scene with virtual camera identical to real camera 2

3 Compositing of synthetic objects Lighting consistency subproblem 1: the synthetic objects need to be lit by the natural light sources that affect the real scene subproblem 2: the synthetic objects affect the real scene 3

4 1. Illumination of synthetic objects with real light Solution 1 modeling the real light sources light source surveying photographing reference object analysis of photographs Solution 2 recording scene illumination using high dynamic range photography 4

5 High dynamic range photography Natural light requires many more intensity levels than the usual 256 More levels can be acquired by varying exposure times Cameras are fast enough to avoid saturation of the brightest natural light sources 5

6 2. Scene affected by synthetic Shadows objects synthetic objects should cast shadows in the scene Reflections synthetic objects should be reflected by shiny scene objects synthetic objects should cast light in the scene 6

7 Overview Introduction Illumination of synthetic objects Complete method for integrating synthetic objects in natural scenes 7

8 Overview Introduction Illumination of synthetic objects Complete method for integrating synthetic objects in natural scenes 8

9 Recording light using radiance maps Radiance: energy of radiation per unit transverse area, in a given direction, of a source of radiation <W sr-1 m-2> Radiance map: in a point, measure radiance in all directions Practical construction: panorama centered in that point or using spherical or parabolic mirrors 9

10 Radiance map example 10

11 Radiance map example 11

12 Radiance map example 12

13 Inter-register exposures to obtain high range radiance map obtained by photographing polished steel sphere (light probe) 13

14 Illuminate synthetic object using a global illumination algorithm 14

15 Another example 15

16 Another example 16

17 Light probe distortions Not all directions are treated equally poor sampling close to the half-sphere edge Camera acquiring the light-probe image can image itself Solution is to take several images of the light probe additional difficulty of interregistering the images 17

18 Reducing distortions using two light-probe images 18

19 Videos 19

20 Overview Introduction Illumination of synthetic objects Complete method for integrating synthetic objects in natural scenes 20

21 Overview Introduction Illumination of synthetic objects Complete method for integrating synthetic objects in natural scenes 21

22 The general method 22

23 Step 1: background acquisition 23

24 Step 2: light probe 24

25 Step 3: constr. the light-based model 25

26 Step 4: camera calibration 26

27 Step 5: global illumination solution 27

28 Step 7: compositing a. render local scene w/o synthetic objects 28

29 Step 7: compositing b. create synthetic objects mask 29

30 Step 7: compositing c. isolate synthetic objects contribution to local scene subtract apply mask 30

31 add Step 7: compositing d. add to the background the synthetic objects and their contribution to the local scene add 31

32 Results: before and after 32