Advanced Digital Photography and Geometry Capture. Visual Imaging in the Electronic Age Lecture #10 Donald P. Greenberg September 24, 2015

Transcription

1 Advanced Digital Photography and Geometry Capture Visual Imaging in the Electronic Age Lecture #10 Donald P. Greenberg September 24, 2015

2 Eye of a Fly AWARE-2 Duke University

3 Insect-Eye Camera MAVs

4 Sony s Curved Sensor 2014

5 Canon s 250-megapixel camera sensor 09/08/15 Can read letters on an airplane 11 miles away!

6 Al Molnar, Cornell University John Wallace, LaserFocusWorld, 4/3/2012.

7 Al Molnar, Cornell University

8 Depth Imaging Original Goals Uses for: Military Defense Security and surveillance

9 Additional Uses of Depth Imaging Gestural Interfaces Cheap 3D scanners Motion capture Bio-metrics Face recognition Gait analysis Skeletonization Manipulating 3D models

10 Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods Time of Flight

11 Camera Definition Camera View direction Model The camera location, view direction, and frustum must be defined relative to the object.

12 Viewing Frustum - User Parameters Location and Orientation (vectors): Eye location of eye point At a point along view direction vector Up a vector defining the Y axis on the hither plane fov Up Shape (scalars): Eye fov field of view angle aspect view plane aspect ratio hither perpendicular distance to hither plane yon perpendicular distance to yon plane hither yon At aspect = width / height

13 In general, what are the unknown variables? Unknown observe position- Xe, Ye, Ze Unknown viewer direction- Φ, θ, Ψ Unknown focal length- f

14 Simple case Known camera positions (x e, y e, z e ), camera optics, Identify corresponding points in each image. Jeremiah Fairbank. View dependent perspective images. Master's thesis, Cornell University, August 2005.

15 Early Work

16 Sagan House

17 1 2 3 Catch Autodesk

18 Autodesk s Remake 2016 Daan Claessen

19 Capturing Geometry from Photographs Can we reconstruct the 3D geometry from an arbitrary set of photographs?

20 Reconstructing Rome 1 The advent of digital photography and the recent growth of photo-sharing websites ( ) have brought about the seismic change in photography and the use of photo collections. 1 A search for the word Rome on returns two million photos. This collection, or others like it, capture every popular site, facade, statue, fountain, interior, café, etc.

21 Characteristics of Typical Photo Sets The photos are unstructured No particular order or distribution of camera viewpoints The photos are uncalibrated Nothing is known about the camera settings (exposure, focal length, etc.) The scale is enormous (millions, not thousands of photos) and We need to do this fast!

22 Correspondence and 3D Structure from Different Camera Positions Note: The pictures are in correspondence 2D dots with same color correspond to the same 3D points.

23 3D Structure from Different Camera Positions Camera 1 Camera 2 Camera 3

24 3D Structure from Different Camera Positions Camera 3 Camera 1 Camera 2 Assuming the position of the red dot is known, there is reprojection error in Camera 3.

25 Change the Problem to an optimization problem Minimize the sum of the squares of the reprojection errors. This non-linear least squares problem is difficult to solve due to local minima and maxima. Authors selectively started with a few choice cameras and points and grew scenes incrementally (a process known as bundle adjustment).

26 Trevi Fountain, Rome Italy

27 Feature Detection and Matching The position and orientation of scale-invariant feature transform (SIFT) features on an image of the Trevi Fountain. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.

28 Feature Detection and Matching A track corresponding to a point on the face of the central statue of Oceanus at the Trevi Fountain, the embodiment of a river encircling the world in Greek mythology. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.

29 Colosseum The Colosseum (Rome) Reconstructed dense 3D point models. For places with many available images, reconstruction quality is very high. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.

30 Cornell Campus, McGraw Hall - Noah Snavely

31 Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods

32 Cyberware Scanner Diagram

33 Cyberware Scanner

34 Uncle Don

35 Cyberware vs Catch

36 Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods

37 Microsoft s Kinect Making Things See, Greg Borenstein

38 Structured Light Imaging (Kinect) Kinect uses a spatial pseudo random neighborhood pattern with unique coding with different sized dots.

39 Kinect speckle pattern Near region ( m) Small dots Middle region ( m) Far region ( m) Large dots

40 Point Cloud Drawn from the Kinect s 3D data

41 Depth Image The gray scale indicates the depth of the object for each pixel. Black indicates no depth information because closer objects obscure those that are further away.

42 Kinect: Depth Image and Real Image

43 Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods Time of Flight

44 Pulsed Modulation

45 Matterport

46 Matterport

47 Matterport 2016

48 Matterport

49 Floored

50 end