3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller
|
|
- Dortha Chandler
- 6 years ago
- Views:
Transcription
1 3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University DFKI Deutsches Forschungszentrum für Künstliche Intelligenz 1
2 Content Motivation Depth Measurement Techniques Applications - Kinect Fusion - Body Reconstruction 2
3 What is a depth camera? A depth camera captured depth images. A depth image indicates in each pixel the distance from the camera to the seen object. (x,y,z) Color Image Depth Image (color encoded) (x,y) In the following slides: z indicates the depth How did we capture depth in the previous lectures. Camera Center 3
4 Depth from Stereo Images image 1 image 2 Dense disparity map Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 4
5 Depth from Stereo Images Goal: recover depth by finding image coordinate x that corresponds to x X X x x x z x' f f C Baseline B C Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 5
6 Stereo and the Epipolar constraint X X X x x x x Potential matches for x have to lie on the corresponding line l. Potential matches for x have to lie on the corresponding line l. Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 6
7 Simplest Case: Parallel images Image planes of cameras are parallel to each other and to the baseline Camera centers are at same height Focal lengths are the same Then, epipolar lines fall along the horizontal scan lines of the images Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 7
8 Basic stereo matching algorithm For each pixel in the first image Find corresponding epipolar line in the right image Examine all pixels on the epipolar line and pick the best match Triangulate the matches to get depth information Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 8
9 disparity Depth from disparity x x O O x f z x B f z X x x f f Baseline B O O z Disparity is inversely proportional to depth! Parts of this slide are adapted from Derek Hoiem (University of Illinois), Steve Seitz (University of Washington) and Lana Lazebnik (University of Illinois) 9
10 Depth Measurement Techniques 10
11 Depth Measurement Techniques Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 11
12 Depth Measurement Techniques Laser Scanner Structured Light Projection Time of Flight (ToF) 12
13 Structured Light Projection Souce: Parts of this slide are adapted from Derek Hoiem (University of Illinois) 13
14 Structured Light Projection (see also lectures about structured light) Surface Projector Sensor Parts of this slide are adapted from Derek Hoiem (University of Illinois) 14
15 Structured Light Projection Projector Camera Parts of this slide are adapted from Derek Hoiem (University of Illinois) 15
16 Example: Book vs. No Book Source: 16
17 Example: Book vs. No Book Source: 17
18 Region-growing Random Dot Matching 1. Detect dots ( speckles ) and label them as unknown 2. Randomly select a region anchor, a dot with unknown depth a. Windowed search via normalized cross correlation along scanline Check that best match score is greater than threshold; if not, mark as invalid and go to 2 b. Region growing 1. Neighboring pixels are added to a queue 2. For each pixel in queue, initialize by anchor s shift; then search small local neighborhood; if matched, add neighbors to queue 3. Stop when no pixels are left in the queue 3. Stop when all dots have known depth or are marked invalid Parts of this slide are adapted from Derek Hoiem (University of Illinois) 18
19 Projected IR vs. Natural Light Stereo What are the advantages of IR? Works in low light conditions Does not rely on having textured objects Not confused by repeated scene textures Can tailor algorithm to produced pattern What are advantages of natural light? Works outside, anywhere with sufficient light Uses less energy Resolution limited only by sensors, not projector Difficulties with both Very dark surfaces may not reflect enough light Specular reflection in mirrors or metal causes trouble Parts of this slide are adapted from Derek Hoiem (University of Illinois) 19
20 Example: The Kinect Sensor (v1) Microsoft Kinect (v1) was released in 2011 as a new kind of controller for the Xbox 360. Parts of this slide are adapted from Rob Miles (University of Hull) 20
21 Example: The Kinect Sensor The Kinect is able to capture depth and color images. Therefore it contains two cameras and an infrared projector. It has also four microphones. Parts of this slide are adapted from Rob Miles (University of Hull) 21
22 Example: The Kinect Sensor The Kinect sensor contains a high quality video camera which can provide up to 1280x1024 resolution at 30 frames a second. Parts of this slide are adapted from Rob Miles (University of Hull) 22
23 Example: The Kinect Sensor IR Projector IR Camera The Kinect depth sensor uses an IR projector and an IR camera to measure the depth of objects in the scene in front of the sensor. Parts of this slide are adapted from Rob Miles (University of Hull) 23
24 Time of Flight (ToF) Time-of-Flight (ToF) Imaging refers to the process of measuring the depth of a scene by quantifying the changes that an emitted light signal encounters when it bounces back from objects in a scene. Two common principals: Pulsed Modulation Continuous Wave Modulation 24
25 Time of Flight (ToF) Pulsed Modulation Measure distance to a 3D object by measuring the absolute time a light pulse needs to travel from a source into the 3D scene and back, after reflection Speed of light is constant and known, c = m/s Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 25
26 Time of Flight (ToF) Pulsed Modulation Advantages: Direct measurement of time-of-flight High-energy light pulses limit influence of background illumination Illumination and observation directions are collinear Disadvantages: High-accuracy time measurement required Measurement of light pulse return is inexact, due to light scattering Difficulty to generate short light pulses with fast rise and fall times Usable light sources (e.g. lasers) suffer low repetition rates for pulses Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 26
27 Time of Flight (ToF) Continuous Wave Modulation Microsoft Kinect v2 works with this principal Continuous light waves instead of short light pulses Modulation in terms of frequency of sinusoidal waves Detected wave after reflection has shifted phase Phase shift proportional to distance from reflecting surface Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 27
28 Time of Flight (ToF) Continuous Wave Modulation Microsoft Kinect v2 works with this principal Retrieve phase shift by demodulation of received signal Demodulation by cross-correlation of received signal with emitted signal Emitted sinusoidal signal: Received signal after reflection from 3D surface: Cross-correlation of both signals: Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 28
29 Time of Flight (ToF) Microsoft Kinect v2 works with this principal Continuous Wave Modulation Cross-correlation function simplifies to Sample at four sequential instants with different phase offset : Directly obtain sought parameters: Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 29
30 Time of Flight (ToF) Microsoft Kinect v2 works with this principal Continuous Wave Modulation Advantages: Variety of light sources available as no short/strong pulses required Applicable to different modulation techniques (other than frequency) Simultaneous range and amplitude images Disadvantages: In practice, integration over time required to reduce noise Frame rates limited by integration time Motion blur caused by long integration time Parts of this slide are adapted from Victor Castaneda and Nassir Navab (both University of Munich) 30
31 Depth Quality e.g. Kinect v1 Souce: Main problems: Resolution Noise 31
32 Depth Camera - Disadvantage: 1. High noise (+/-15mm) 2. Low resolution (176*144) 3. High distortion + Advantage: 1. Real-time capture 2. Video frame with 2/3D information Variance distribution in a depth image taken at approx. 1.5m average distance from a scene. Depth images contain heavy noise near the corners.
33 Applications Kinect Fusion Body Reconstruction 34
34 Kinect Fusion Paper link (ACM Symposium on User Interface Software and Technology, October 2011) YouTube Video
35 Challenges Tracking camera precisely Fusing and de-noising measurements (depth estimates) Avoiding drift Real-Time Low-Cost hardware Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 37
36 Proposed Solution Fast optimization for tracking; due to high frame rate Global framework for fusing data Interleaving tracking & mapping Using Kinect to get depth data ( low cost) Using GPU to get real-time performance ( low cost) Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 38
37 Method Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 39
38 KinectFusion- Depth map L projects a light spot P on an object surface, and O observes the spot Triangle (OPL) solve d Standard structured lighting model Given b,α and β Depth image (VGA &11-bit ) 40/72
39 KinectFusion-Vertex and normal map Vertex map is a 3D point cloud 3D vertex depth 2D depth point intrinsic matrix (IR camera) Normal vector indicates the direction of the surface at a vertex cross product neighboring points 41/72
40 KinectFusion- Camera tracking Small motion between consecutive positions of Kinect Find correspondences using projective data association Estimate camera pose T i by applying ICP algorithm to vertex and normal maps Tracking camera pose 42/72
41 Tracking Finding camera position is the same as fitting the depth map of a frame onto Model Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 43
42 Tracking ICP algorithm ICP = iterative closest point Goal: fit two 3D point sets Already explained in Structured Light lecture Problem: What are the correspondences? Kinect fusion chosen solution: 1) Start with T 0 2) Project model onto camera 3) Correspondences are points with same coordinates 4) Find new T with Least Squares (with the 3D-3D points) 5) Apply T, and repeat 2-5 until convergence Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 44
43 Tracking ICP algorithm Tracking Mapping Assumption: frame and model are roughly aligned. True because of high frame rate Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 45
44 Mapping Mapping is fusing depth maps when camera poses are known Problems: measurements are noisy Depth maps have holes Solution: Using implicit surface representation Fusing = estimations from all frames relevant Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 46
45 Mapping surface representation Surface is represented implicitly using Truncated Signed Distance Function (TSDF) Voxel grid Tracking Mapping Numbers in cells measure voxel distance to surface Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 47
46 KinectFusion- Volumetric integration Volumetric representation (3 3 3m, 512 voxels/axis) (0,1] (outside of the surface) tsdf(g) = 0 (on the surface) [-1,0) (inside the surface) TSDF volume grid 48/72
47 Mapping Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 49
48 Mapping Tracking Mapping d= [pixel depth] [distance from sensor to voxel] Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 50
49 Mapping Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 51
50 Mapping Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 52
51 Mapping Tracking Mapping Last step: Voxel D is the weighted average of all measurements Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 53
52 Handling drift Drift would have happened, if tracking was done from frame to frame Thus, tracking is done on built model Tracking Mapping Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 55
53 KinectFusion- Surface rendering Ray-casting technique Cast a ray through the focal point for each pixel Traverse voxels along the ray Find the first surface by observing the sign change of tsdf(g) Compute the intersection point using points around the surface boundary surface YouTube Video TSDF volume grid 56/72
54 Pros: Pros & Cons Nice results Real time performance (30 Hz) Dense model No drift with local optimization Elegant solution Cons : 3D grid can not be trivially up-scaled Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 57
55 Limitations Doesn t work for large areas (Voxel-Grid) Doesn t work far away from objects (active ranging) Doesn t work well out-doors (IR) Requires powerful graphics card Uses lots of battery (active ranging) Parts of this slide are adapted from Richard A. Newcombe (Imperial College London) and Boaz Petersil (Israel Institute of Technology) 58
56
57 Thank you!
Depth Sensors Kinect V2 A. Fornaser
Depth Sensors Kinect V2 A. Fornaser alberto.fornaser@unitn.it Vision Depth data It is not a 3D data, It is a map of distances Not a 3D, not a 2D it is a 2.5D or Perspective 3D Complete 3D - Tomography
More informationDepth Cameras. Didier Stricker Oliver Wasenmüller Lecture 3D Computer Vision
1 Depth Cameras Lecture 3D Computer Vision Oliver Wasenmüller oliver.wasenmueller@dfki.de Didier Stricker didier.stricker@dfki.de Content Motivation Depth Measurement Techniques Depth Image Enhancement
More informationKinect Device. How the Kinect Works. Kinect Device. What the Kinect does 4/27/16. Subhransu Maji Slides credit: Derek Hoiem, University of Illinois
4/27/16 Kinect Device How the Kinect Works T2 Subhransu Maji Slides credit: Derek Hoiem, University of Illinois Photo frame-grabbed from: http://www.blisteredthumbs.net/2010/11/dance-central-angry-review
More informationLecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013
Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth
More informationHuman Body Recognition and Tracking: How the Kinect Works. Kinect RGB-D Camera. What the Kinect Does. How Kinect Works: Overview
Human Body Recognition and Tracking: How the Kinect Works Kinect RGB-D Camera Microsoft Kinect (Nov. 2010) Color video camera + laser-projected IR dot pattern + IR camera $120 (April 2012) Kinect 1.5 due
More information3D Computer Vision. Structured Light I. Prof. Didier Stricker. Kaiserlautern University.
3D Computer Vision Structured Light I Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction
More informationCS4495/6495 Introduction to Computer Vision
CS4495/6495 Introduction to Computer Vision 9C-L1 3D perception Some slides by Kelsey Hawkins Motivation Why do animals, people & robots need vision? To detect and recognize objects/landmarks Is that a
More information3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University.
3D Computer Vision Structured Light II Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationBinocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?
Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationStereo vision. Many slides adapted from Steve Seitz
Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is
More informationImage Based Reconstruction II
Image Based Reconstruction II Qixing Huang Feb. 2 th 2017 Slide Credit: Yasutaka Furukawa Image-Based Geometry Reconstruction Pipeline Last Lecture: Multi-View SFM Multi-View SFM This Lecture: Multi-View
More information3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava
3D Computer Vision Dense 3D Reconstruction II Prof. Didier Stricker Christiano Gava Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationMultiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
More informationStereo. Many slides adapted from Steve Seitz
Stereo Many slides adapted from Steve Seitz Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image image 1 image 2 Dense depth map Binocular stereo Given a calibrated
More information3D Photography: Active Ranging, Structured Light, ICP
3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring 2013 http://cvg.ethz.ch/teaching/2013spring/3dphoto/ Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar
More informationGeometric Reconstruction Dense reconstruction of scene geometry
Lecture 5. Dense Reconstruction and Tracking with Real-Time Applications Part 2: Geometric Reconstruction Dr Richard Newcombe and Dr Steven Lovegrove Slide content developed from: [Newcombe, Dense Visual
More informationOutline. ETN-FPI Training School on Plenoptic Sensing
Outline Introduction Part I: Basics of Mathematical Optimization Linear Least Squares Nonlinear Optimization Part II: Basics of Computer Vision Camera Model Multi-Camera Model Multi-Camera Calibration
More informationDepth Camera for Mobile Devices
Depth Camera for Mobile Devices Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Stereo Cameras Structured Light Cameras Time of Flight (ToF) Camera Inferring 3D Points Given we have
More informationRange Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
More informationOutline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping
Outline CSE 576 KinectFusion: Real-Time Dense Surface Mapping and Tracking PhD. work from Imperial College, London Microsoft Research, Cambridge May 6, 2013 1 Why we re interested in Real-Time tracking
More information3D Photography: Stereo
3D Photography: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://www.cvg.ethz.ch/teaching/3dvision/ 3D Modeling with Depth Sensors Today s class Obtaining depth maps / range images unstructured
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationStructured Light. Tobias Nöll Thanks to Marc Pollefeys, David Nister and David Lowe
Structured Light Tobias Nöll tobias.noell@dfki.de Thanks to Marc Pollefeys, David Nister and David Lowe Introduction Previous lecture: Dense reconstruction Dense matching of non-feature pixels Patch-based
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationKinectFusion: Real-Time Dense Surface Mapping and Tracking
KinectFusion: Real-Time Dense Surface Mapping and Tracking Gabriele Bleser Thanks to Richard Newcombe for providing the ISMAR slides Overview General: scientific papers (structure, category) KinectFusion:
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationChaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationStereo Vision A simple system. Dr. Gerhard Roth Winter 2012
Stereo Vision A simple system Dr. Gerhard Roth Winter 2012 Stereo Stereo Ability to infer information on the 3-D structure and distance of a scene from two or more images taken from different viewpoints
More informationMulti-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
More informationTime-of-Flight Imaging!
Time-of-Flight Imaging Loren Schwarz, Nassir Navab 3D Computer Vision II Winter Term 2010 21.12.2010 Lecture Outline 1. Introduction and Motivation 2. Principles of ToF Imaging 3. Computer Vision with
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationCameras and Stereo CSE 455. Linda Shapiro
Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation
More informationMulti-View 3D-Reconstruction
Multi-View 3D-Reconstruction Cedric Cagniart Computer Aided Medical Procedures (CAMP) Technische Universität München, Germany 1 Problem Statement Given several calibrated views of an object... can we automatically
More informationMiniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
More informationStructured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov
Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter
More informationLecture'9'&'10:'' Stereo'Vision'
Lecture'9'&'10:'' Stereo'Vision' Dr.'Juan'Carlos'Niebles' Stanford'AI'Lab' ' Professor'FeiAFei'Li' Stanford'Vision'Lab' 1' Dimensionality'ReducIon'Machine'(3D'to'2D)' 3D world 2D image Point of observation
More information3D Scanning. Qixing Huang Feb. 9 th Slide Credit: Yasutaka Furukawa
3D Scanning Qixing Huang Feb. 9 th 2017 Slide Credit: Yasutaka Furukawa Geometry Reconstruction Pipeline This Lecture Depth Sensing ICP for Pair-wise Alignment Next Lecture Global Alignment Pairwise Multiple
More information3D Computer Vision 1
3D Computer Vision 1 Multiview Stereo Multiview Stereo Multiview Stereo https://www.youtube.com/watch?v=ugkb7itpnae Shape from silhouette Shape from silhouette Shape from silhouette Shape from silhouette
More information3D Object Representations. COS 526, Fall 2016 Princeton University
3D Object Representations COS 526, Fall 2016 Princeton University 3D Object Representations How do we... Represent 3D objects in a computer? Acquire computer representations of 3D objects? Manipulate computer
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationEECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems
EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationLecture 14: Computer Vision
CS/b: Artificial Intelligence II Prof. Olga Veksler Lecture : Computer Vision D shape from Images Stereo Reconstruction Many Slides are from Steve Seitz (UW), S. Narasimhan Outline Cues for D shape perception
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More informationLaser sensors. Transmitter. Receiver. Basilio Bona ROBOTICA 03CFIOR
Mobile & Service Robotics Sensors for Robotics 3 Laser sensors Rays are transmitted and received coaxially The target is illuminated by collimated rays The receiver measures the time of flight (back and
More informationRange Sensors (time of flight) (1)
Range Sensors (time of flight) (1) Large range distance measurement -> called range sensors Range information: key element for localization and environment modeling Ultrasonic sensors, infra-red sensors
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationMobile Point Fusion. Real-time 3d surface reconstruction out of depth images on a mobile platform
Mobile Point Fusion Real-time 3d surface reconstruction out of depth images on a mobile platform Aaron Wetzler Presenting: Daniel Ben-Hoda Supervisors: Prof. Ron Kimmel Gal Kamar Yaron Honen Supported
More informationStereo: Disparity and Matching
CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS2 is out. But I was late. So we pushed the due date to Wed Sept 24 th, 11:55pm. There is still *no* grace period. To
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationHierarchical Volumetric Fusion of Depth Images
Hierarchical Volumetric Fusion of Depth Images László Szirmay-Kalos, Milán Magdics Balázs Tóth, Tamás Umenhoffer Real-time color & 3D information Affordable integrated depth and color cameras Application:
More information10/5/09 1. d = 2. Range Sensors (time of flight) (2) Ultrasonic Sensor (time of flight, sound) (1) Ultrasonic Sensor (time of flight, sound) (2) 4.1.
Range Sensors (time of flight) (1) Range Sensors (time of flight) (2) arge range distance measurement -> called range sensors Range information: key element for localization and environment modeling Ultrasonic
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Stereo Vision 2 Inferring 3D from 2D Model based pose estimation single (calibrated) camera Stereo
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationDense Tracking and Mapping for Autonomous Quadrocopters. Jürgen Sturm
Computer Vision Group Prof. Daniel Cremers Dense Tracking and Mapping for Autonomous Quadrocopters Jürgen Sturm Joint work with Frank Steinbrücker, Jakob Engel, Christian Kerl, Erik Bylow, and Daniel Cremers
More informationPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Teesta suspension bridge-darjeeling, India Mark Twain at Pool Table", no date, UCR Museum of Photography Woman getting eye exam during
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationCS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017
CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo
More informationActive Stereo Vision. COMP 4900D Winter 2012 Gerhard Roth
Active Stereo Vision COMP 4900D Winter 2012 Gerhard Roth Why active sensors? Project our own texture using light (usually laser) This simplifies correspondence problem (much easier) Pluses Can handle different
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More informationComplex Sensors: Cameras, Visual Sensing. The Robotics Primer (Ch. 9) ECE 497: Introduction to Mobile Robotics -Visual Sensors
Complex Sensors: Cameras, Visual Sensing The Robotics Primer (Ch. 9) Bring your laptop and robot everyday DO NOT unplug the network cables from the desktop computers or the walls Tuesday s Quiz is on Visual
More informationLecture 10 Dense 3D Reconstruction
Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,
More informationA Comparison between Active and Passive 3D Vision Sensors: BumblebeeXB3 and Microsoft Kinect
A Comparison between Active and Passive 3D Vision Sensors: BumblebeeXB3 and Microsoft Kinect Diana Beltran and Luis Basañez Technical University of Catalonia, Barcelona, Spain {diana.beltran,luis.basanez}@upc.edu
More informationc 2014 Gregory Paul Meyer
c 2014 Gregory Paul Meyer 3D FACE MODELING WITH A CONSUMER DEPTH CAMERA BY GREGORY PAUL MEYER THESIS Submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical
More informationImage Rectification (Stereo) (New book: 7.2.1, old book: 11.1)
Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Stereo Vision 2 Inferring 3D from 2D Model based pose estimation single (calibrated) camera > Can
More informationLecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationProcessing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 17 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
More informationVisual Perception Sensors
G. Glaser Visual Perception Sensors 1 / 27 MIN Faculty Department of Informatics Visual Perception Sensors Depth Determination Gerrit Glaser University of Hamburg Faculty of Mathematics, Informatics and
More informationMesh from Depth Images Using GR 2 T
Mesh from Depth Images Using GR 2 T Mairead Grogan & Rozenn Dahyot School of Computer Science and Statistics Trinity College Dublin Dublin, Ireland mgrogan@tcd.ie, Rozenn.Dahyot@tcd.ie www.scss.tcd.ie/
More informationSimultaneous Localization and Mapping (SLAM)
Simultaneous Localization and Mapping (SLAM) RSS Lecture 16 April 8, 2013 Prof. Teller Text: Siegwart and Nourbakhsh S. 5.8 SLAM Problem Statement Inputs: No external coordinate reference Time series of
More informationThe main problem of photogrammetry
Structured Light Structured Light The main problem of photogrammetry to recover shape from multiple views of a scene, we need to find correspondences between the images the matching/correspondence problem
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More informationSensor Modalities. Sensor modality: Different modalities:
Sensor Modalities Sensor modality: Sensors which measure same form of energy and process it in similar ways Modality refers to the raw input used by the sensors Different modalities: Sound Pressure Temperature
More informationVolume Illumination, Contouring
Volume Illumination, Contouring Computer Animation and Visualisation Lecture 0 tkomura@inf.ed.ac.uk Institute for Perception, Action & Behaviour School of Informatics Contouring Scaler Data Overview -
More informationProcessing 3D Surface Data
Processing 3D Surface Data Computer Animation and Visualisation Lecture 12 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing
More informationLecture 16: Computer Vision
CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field
More informationLecture 16: Computer Vision
CS442/542b: Artificial ntelligence Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field Methods
More informationIEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 6, NO. 5, SEPTEMBER
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 6, NO. 5, SEPTEMBER 2012 411 Consistent Stereo-Assisted Absolute Phase Unwrapping Methods for Structured Light Systems Ricardo R. Garcia, Student
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationEpipolar geometry contd.
Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives
More informationIntroduction to 3D Machine Vision
Introduction to 3D Machine Vision 1 Many methods for 3D machine vision Use Triangulation (Geometry) to Determine the Depth of an Object By Different Methods: Single Line Laser Scan Stereo Triangulation
More informationSensing Deforming and Moving Objects with Commercial Off the Shelf Hardware
Sensing Deforming and Moving Objects with Commercial Off the Shelf Hardware This work supported by: Philip Fong Florian Buron Stanford University Motivational Applications Human tissue modeling for surgical
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More information3D Modeling of Objects Using Laser Scanning
1 3D Modeling of Objects Using Laser Scanning D. Jaya Deepu, LPU University, Punjab, India Email: Jaideepudadi@gmail.com Abstract: In the last few decades, constructing accurate three-dimensional models
More informationStereo and structured light
Stereo and structured light http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 20 Course announcements Homework 5 is still ongoing. - Make sure
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More information