Advanced Digital Photography and Geometry Capture Visual Imaging in the Electronic Age Lecture #10 Donald P. Greenberg September 24, 2015
Eye of a Fly AWARE-2 Duke University http://www.nanowerk.com/spotlight/spotid=3744.php
Insect-Eye Camera MAVs http://spectrum.ieee.org/robotics/robotics-hardware/insecteye-camera-offers-wideangle-vision-for-tiny-drones
Sony s Curved Sensor 2014
Canon s 250-megapixel camera sensor 09/08/15 Can read letters on an airplane 11 miles away!
Al Molnar, Cornell University John Wallace, LaserFocusWorld, 4/3/2012.
Al Molnar, Cornell University
Depth Imaging Original Goals Uses for: Military Defense Security and surveillance
Additional Uses of Depth Imaging Gestural Interfaces Cheap 3D scanners Motion capture Bio-metrics Face recognition Gait analysis Skeletonization Manipulating 3D models
Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods Time of Flight
Camera Definition Camera View direction Model The camera location, view direction, and frustum must be defined relative to the object.
Viewing Frustum - User Parameters Location and Orientation (vectors): Eye location of eye point At a point along view direction vector Up a vector defining the Y axis on the hither plane fov Up Shape (scalars): Eye fov field of view angle aspect view plane aspect ratio hither perpendicular distance to hither plane yon perpendicular distance to yon plane hither yon At aspect = width / height
In general, what are the unknown variables? Unknown observe position- Xe, Ye, Ze Unknown viewer direction- Φ, θ, Ψ Unknown focal length- f
Simple case Known camera positions (x e, y e, z e ), camera optics, Identify corresponding points in each image. Jeremiah Fairbank. View dependent perspective images. Master's thesis, Cornell University, August 2005.
Early Work - 1975
Sagan House
1 2 3 Catch Autodesk
Autodesk s Remake 2016 Daan Claessen
Capturing Geometry from Photographs Can we reconstruct the 3D geometry from an arbitrary set of photographs?
Reconstructing Rome 1 The advent of digital photography and the recent growth of photo-sharing websites ( ) have brought about the seismic change in photography and the use of photo collections. 1 A search for the word Rome on returns two million photos. This collection, or others like it, capture every popular site, facade, statue, fountain, interior, café, etc.
Characteristics of Typical Photo Sets The photos are unstructured No particular order or distribution of camera viewpoints The photos are uncalibrated Nothing is known about the camera settings (exposure, focal length, etc.) The scale is enormous (millions, not thousands of photos) and We need to do this fast!
Correspondence and 3D Structure from Different Camera Positions Note: The pictures are in correspondence 2D dots with same color correspond to the same 3D points.
3D Structure from Different Camera Positions Camera 1 Camera 2 Camera 3
3D Structure from Different Camera Positions Camera 3 Camera 1 Camera 2 Assuming the position of the red dot is known, there is reprojection error in Camera 3.
Change the Problem to an optimization problem Minimize the sum of the squares of the reprojection errors. This non-linear least squares problem is difficult to solve due to local minima and maxima. Authors selectively started with a few choice cameras and points and grew scenes incrementally (a process known as bundle adjustment).
Trevi Fountain, Rome Italy http://en.wikipedia.org/wiki/trevi_fountain
Feature Detection and Matching The position and orientation of scale-invariant feature transform (SIFT) features on an image of the Trevi Fountain. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.
Feature Detection and Matching A track corresponding to a point on the face of the central statue of Oceanus at the Trevi Fountain, the embodiment of a river encircling the world in Greek mythology. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.
Colosseum The Colosseum (Rome) Reconstructed dense 3D point models. For places with many available images, reconstruction quality is very high. Sameer Agarwal, Yasutaka Furukawa, Naoh Snavely, Brian Curless, Steve M. Seitz, Richard Szeliski. Reconstructing Rome, IEEE Computer, June 2010.
Cornell Campus, McGraw Hall - Noah Snavely
Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods
Cyberware Scanner Diagram
Cyberware Scanner
Uncle Don
Cyberware vs. 1 2 3 Catch
Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods
Microsoft s Kinect Making Things See, Greg Borenstein
Structured Light Imaging (Kinect) Kinect uses a spatial pseudo random neighborhood pattern with unique coding with different sized dots.
Kinect speckle pattern Near region (0.8 1.2m) Small dots Middle region (1.2 2.0m) Far region (2.0 3.5 m) Large dots
Point Cloud Drawn from the Kinect s 3D data
Depth Image The gray scale indicates the depth of the object for each pixel. Black indicates no depth information because closer objects obscure those that are further away.
Kinect: Depth Image and Real Image
Digital Geometry Capture Photographic methods Laser scanning Pattern projection methods Time of Flight
Pulsed Modulation
Matterport
Matterport
Matterport 2016
Matterport
Floored http://labs.floored.com/clients/mrp-realty/900-g-roof/
end