Dense 3D Reconstruction. Christiano Gava
|
|
- Sherman Carter
- 5 years ago
- Views:
Transcription
1 Dense 3D Reconstruction Christiano Gava
2 Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction The matching problem 2-view reconstruction Multi-view reconstruction Next lecture: structured light 12/11/2012 Lecture 3D Computer Vision 2
3 Why dense reconstruction? Accurate 3D models cultural heritage! Reconstruction of houses, buildings, famous touristic sites Piazza San Marco Venice images points Interesting! But how can we go from pictures to 3D models? 12/11/2012 Lecture 3D Computer Vision 3
4 Overview I 12/11/2012 Lecture 3D Computer Vision 4
5 Overview II 12/11/2012 Lecture 3D Computer Vision 5
6 Dense 3D reconstruction When we take a picture, a 3D scene is projected onto a 2D image loss of depth information 3D reconstruction is the inverse process: build the 3D scene from a set of 2D images recover depth information How can we do that? MATCHING (difficult) + TRIANGULATION (previous lecture) 12/11/2012 Lecture 3D Computer Vision 6
7 Two Subproblems In general, 3D reconstruction can be divided into two major problems: Matching finding corresponding elements among the images Triangulation establishing 3D coordinates based on 2D image correspondences Which image entities should we match? 12/11/2012 Lecture 3D Computer Vision 7
8 Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction The matching problem 2-view reconstruction Multi-view reconstruction Next lecture: structured light 12/11/2012 Lecture 3D Computer Vision 8
9 Matching Harris corners Lines/edges Feature-based matching SIFT keypoints, etc. Suitable for Structure and Motion But we are interested in dense reconstruction we need dense matching! match as many pixels as possible 12/11/2012 Lecture 3D Computer Vision 9
10 Dense matching Choice of camera setup Small baseline X wide baseline Matching is relatively easy Higher depth uncertainty Matching is hard Lower depth uncertainty 12/11/2012 Lecture 3D Computer Vision 10
11 Dense matching For every point on an image, there are many possible matches on the other image(s) Many points look similar high ambiguity What can we do? 12/11/2012 Lecture 3D Computer Vision 11
12 Dense matching We can use camera geometry finding the correspondence becomes a 1D search problem thanks to the epipolar geometry 12/11/2012 Lecture 3D Computer Vision 12
13 Dense matching Even using the epipolar constraint, there are many possible matches? Use of pixel neighborhood information correlation-based approaches (also called window-based approaches) 12/11/2012 Lecture 3D Computer Vision 13
14 Common window-based approaches Normalized Cross-Correlation (NCC) Sum of Squared Differences (SSD) Sum of Absolute Differences (SAD) 12/11/2012 Lecture 3D Computer Vision 14
15 Recall NCC write regions as vectors region A region B a b vector a vector b 12/11/2012 Lecture 3D Computer Vision 15
16 Window-based matching Search along the epipolar line for the best match Best match: - Maximum NCC - Minimum SSD or SAD 12/11/2012 Lecture 3D Computer Vision 16
17 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/11/2012 Lecture 3D Computer Vision 17
18 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 18
19 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 19
20 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 20
21 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 21
22 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 22
23 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 23
24 Dense matching challenges Scene elements do not always look the same in the images Camera-related problems Image noise Lens distortion Color/chromatic aberration Viewpoint-related problems Perspective distortions Occlusions Specular reflections Scene-related problems Illumination changes Moving objects 12/12/2012 Lecture 3D Computer Vision 24
25 Dense matching challenges Matching ambiguity Low textured regions Repetitive texture pattern Some assumptions are made 12/11/2012 Lecture 3D Computer Vision 25
26 Assumptions Appearance The projections of a scene (3D) patch should have similar appearances in all images Uniqueness A point in an image will have only one match in another image Ordering Order of appearance (e.g. left to right) is not altered by camera displacement Smoothness Depth varies smoothly (objects have smooth surfaces) 12/11/2012 Lecture 3D Computer Vision 26
27 Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction The matching problem 2-view reconstruction Multi-view reconstruction Next lecture: structured light 12/11/2012 Lecture 3D Computer Vision 27
28 2-view reconstruction Input: a pair of images (usually left and right) Output: a disparity or depth map left image right image depth map disparity 1 depth 12/11/2012 Lecture 3D Computer Vision 28
29 2-view reconstruction General case: converging cameras Need to use epipolar geometry This geometry can be simplified by rectification One of the images Both images 12/11/2012 Lecture 3D Computer Vision 29
30 Image rectification Rectifying one image Image mapping is a 2D homography (projective transformation) Rectifying both images Project images onto plane parallel to baseline epipolar plane 12/11/2012 Lecture 3D Computer Vision 30
31 Example of rectifying both images 12/11/2012 Lecture 3D Computer Vision 31
32 Parallel cameras Triangulation reduces to a simple geometric problem x Z x d 12/11/2012 Lecture 3D Computer Vision 32
33 General dense correspondence algorithm For each pixel in the left image compute the neighbourhood cross correlation along the corresponding epipolar line in the right image the corresponding pixel is the one with the highest cross correlation Parameters size (scale) of neighbourhood search disparity Other constraints uniqueness ordering smoothness of depth field Applicability textured scene, largely fronto-parallel 12/11/2012 Lecture 3D Computer Vision 33
34 Example with NCC epipolar line 12/11/2012 Lecture 3D Computer Vision 34
35 Example with NCC left image band right image band cross correlation 0 12/11/2012 Lecture 3D Computer Vision 35 x
36 Example with NCC target region left image band right image band cross correlation 0 Why is it not as good as before? 12/11/2012 Lecture 3D Computer Vision 36 x
37 2-view reconstruction 1.The neighbourhood region does not have a distinctive spatial intensity distribution 2.Foreshortening effects fronto-parallel surface imaged length the same slanting surface imaged lengths differ
38 Occlusion surface slice , , /11/2012 Lecture 3D Computer Vision 38
39 Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction The matching problem 2-view reconstruction Multi-view reconstruction Next lecture: structured light 12/11/2012 Lecture 3D Computer Vision 39
40 Multi-view reconstruction Many views of the same scene More images More constraints! x 1 x 2 x l 3 31 l 32 l 31 = F T 13 x 1 l 32 = F T 23 x 2 12/11/2012 Lecture 3D Computer Vision 40
41 Multi-view reconstruction More constraints Less occlusions More robust Appearance constraint can be relaxed However More complex Strong perspective distortions Propagation of calibration uncertainties Dense matching along many images is still hard Precise calibration is essential! 12/11/2012 Lecture 3D Computer Vision 41
42 Multi-view triangulation Assuming we have found correct matches along n images, how do we triangulate them? X j Exactly as we did before! Remember the algebraic solution x 1j Stack equations for all camera views: x 3j C 1 x 2j C 2 C 3 12/11/2012 Lecture 3D Computer Vision 42
43 Multi-view reconstruction MATCHING SCENE REPRESENTATION TRIANGULATION 12/11/2012 Lecture 3D Computer Vision 43
44 Scene Representation Voxel grid Depth map Mesh Point cloud 12/11/2012 Lecture 3D Computer Vision 44
45 Divide the scene into cubes Voxel grid Needs to sample the 3D space Voxels are marked as occupied/empty according to a photometric consistency measure (NCC, SSD ) Accuracy x size of voxels No assumption about the scene Memory allocation issues 12/12/2012 Lecture 3D Computer Vision 45
46 Divide the scene into cubes Voxel grid Needs to sample the 3D space Voxels are marked as occupied/empty according to a photometric consistency measure (NCC, SSD ) Accuracy x size of voxels No assumption about the scene Memory allocation issues 12/11/2012 Lecture 3D Computer Vision 46
47 Divide the scene into cubes Voxel grid Needs to sample the 3D space Voxels are marked as occupied/empty according to a photometric consistency measure (NCC, SSD ) Accuracy x size of voxels No assumption about the scene Memory allocation issues 12/12/2012 Lecture 3D Computer Vision 47
48 Divide the scene into cubes Voxel grid Needs to sample the 3D space Voxels are marked as occupied/empty according to a photometric consistency measure (NCC, SSD ) Accuracy x size of voxels No assumption about the scene Memory allocation issues 12/12/2012 Lecture 3D Computer Vision 48
49 Usually one depth map per view (by matching and triangulating features in small sets of images) Depth map No sampling of 3D space Simple Ideal for multi-core Depth maps have to be merged 12/11/2012 Lecture 3D Computer Vision 49
50 Usually one depth map per view (by matching and triangulating features in small sets of images) Depth map No sampling of 3D space Simple Ideal for multi-core Depth maps have to be merged 12/12/2012 Lecture 3D Computer Vision 50
51 Point cloud Flexible Points are independent Global or local Allows divide and conquer Sparse or dense May be noisy 12/11/2012 Lecture 3D Computer Vision 51
52 Point cloud Flexible Points are independent Global or local Allows divide and conquer Sparse or dense May be noisy 12/12/2012 Lecture 3D Computer Vision 52
53 Point cloud Flexible Points are independent Global or local Allows divide and conquer Sparse or dense May be noisy 12/12/2012 Lecture 3D Computer Vision 53
54 Point cloud 12/11/2012 Lecture 3D Computer Vision 54
55 Mesh Models the surface as a connected set of planar facets Usually triangular meshes Triangles are local good approximations of the surface Less noisy Good for optimization (mesh refinement) May become difficult to handle for complex surfaces 12/11/2012 Lecture 3D Computer Vision 55
56 Mesh Models the surface as a connected set of planar facets Usually triangular meshes Triangles are local good approximations of the surface Less noisy Good for optimization (mesh refinement) May become difficult to handle for complex surfaces 12/12/2012 Lecture 3D Computer Vision 56
57 Mesh Models the surface as a connected set of planar facets Usually triangular meshes Triangles are local good approximations of the surface Less noisy Good for optimization (mesh refinement) May become difficult to handle for complex surfaces 12/12/2012 Lecture 3D Computer Vision 57
58 Mesh 12/11/2012 Lecture 3D Computer Vision 58
59 Mesh 12/11/2012 Lecture 3D Computer Vision 59
60 Interesting links Middleburry stereo data set: Middlebury multi-view data set: PMVS version 2: CVLab multi-view data sets: /12/2012 Lecture 3D Computer Vision 60
61 References Furukawa, Y. & Ponce, J. Accurate, Dense and Robust Multi-View Stereopsis, TPAMI, 2008, 01, 1-14 Furukawa, Y.; Curless, B.; Seitz, S. M. & Szeliski, R. Towards Internet- Scale Multi-View Stereo, CVPR, 2010 Hiep, V. H.; Keriven, R.; Labatut, P. & Pons, J.-P. Towards highresolution large-scale multi-view stereo, CVPR, 2009, Goesele, M.; Snavely, N.; Curless, B.; Hoppe, H. & Seitz, S. M. Multi- View Stereo for Community Photo Collections, ICCV, 2007, 0, 1-8 Hartley, R. I. & Zisserman, A. Multiple View Geometry in Computer Vision, Cambridge University Press, ISBN: , /12/2012 Lecture 3D Computer Vision 61
62 Organisational information Oral exams: Qualification: A minimum average score of 60% in the exercises Period: DFKI, room 1.27 Registration: to Leivy Michelly Kaul (kaul@dfki.uni-kl.de) to get an appointment (mention days where you can t make it). Lectures: : no lecture; you are expected to work through a scientific paper based on central questions (will be uploaded to the website shortly before). The paper will be the basis for the lecture at The last lecture will be a question lecture and can be moved from to a date closer to the actual exam period (will be coordinated by Johannes and Tobias in the last exercise session). Lecture 3D Computer Vision 62
63 Thank you!
64 JUST A REMINDER 12/11/2012 Lecture 3D Computer Vision 64
65 Triangulation If images are not rectified X j x 1j x 2j C 1 C 2 12/11/2012 Lecture 3D Computer Vision 65
66 Triangulation Given: corresponding measured (i.e. noisy) feature points x and x and camera poses, P and P Problem: in the presence of noise, back projected rays do not intersect Rays are skew in space and measured points do not lie on corresponding epipolar lines 12/11/2012 Lecture 3D Computer Vision 66
67 Triangulation: algebraic solution Equation for one camera view (camera pose P and feature point x) 4 entries, 3 DOF i th row of P (3x4) matrix 3 equations, only 2 linearly independent drop third row 12/11/2012 Lecture 3D Computer Vision 67
68 Triangulation: algebraic solution Stack equations for all camera views: X has 3 DOF (4 parameters due to homogeneous representation) One camera view gives two linearly independent equations Two camera views are sufficient for a minimal solution Solution for X is eigenvector of A T A corresponding to smallest eigenvalue 68
69 Triangulation: minimize geometric error Estimate 3D point, which exactly satisfies the supplied camera geometry P and P, so it projects as where and are closest to the actual image measurements Assumes perfect camera poses! 12/11/2012 Lecture 3D Computer Vision 69
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More information3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava
3D Computer Vision Dense 3D Reconstruction II Prof. Didier Stricker Christiano Gava Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationMulti-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationApplication questions. Theoretical questions
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationLecture 10 Dense 3D Reconstruction
Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationChaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
More informationLecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,
More informationStereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision
Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras
More informationGeometric Reconstruction Dense reconstruction of scene geometry
Lecture 5. Dense Reconstruction and Tracking with Real-Time Applications Part 2: Geometric Reconstruction Dr Richard Newcombe and Dr Steven Lovegrove Slide content developed from: [Newcombe, Dense Visual
More informationMulti-View Stereo for Static and Dynamic Scenes
Multi-View Stereo for Static and Dynamic Scenes Wolfgang Burgard Jan 6, 2010 Main references Yasutaka Furukawa and Jean Ponce, Accurate, Dense and Robust Multi-View Stereopsis, 2007 C.L. Zitnick, S.B.
More informationCameras and Stereo CSE 455. Linda Shapiro
Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation
More informationFinal project bits and pieces
Final project bits and pieces The project is expected to take four weeks of time for up to four people. At 12 hours per week per person that comes out to: ~192 hours of work for a four person team. Capstone:
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationFundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F
Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationGeometric camera models and calibration
Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationImage Based Reconstruction II
Image Based Reconstruction II Qixing Huang Feb. 2 th 2017 Slide Credit: Yasutaka Furukawa Image-Based Geometry Reconstruction Pipeline Last Lecture: Multi-View SFM Multi-View SFM This Lecture: Multi-View
More informationMulti-stable Perception. Necker Cube
Multi-stable Perception Necker Cube Spinning dancer illusion, Nobuyuki Kayahara Multiple view geometry Stereo vision Epipolar geometry Lowe Hartley and Zisserman Depth map extraction Essential matrix
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationMulti-View 3D-Reconstruction
Multi-View 3D-Reconstruction Cedric Cagniart Computer Aided Medical Procedures (CAMP) Technische Universität München, Germany 1 Problem Statement Given several calibrated views of an object... can we automatically
More informationWhat have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationCS5670: Computer Vision
CS5670: Computer Vision Noah Snavely, Zhengqi Li Stereo Single image stereogram, by Niklas Een Mark Twain at Pool Table", no date, UCR Museum of Photography Stereo Given two images from different viewpoints
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Teesta suspension bridge-darjeeling, India Mark Twain at Pool Table", no date, UCR Museum of Photography Woman getting eye exam during
More informationDepth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
More informationStereo vision. Many slides adapted from Steve Seitz
Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is
More informationStructure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
More informationStereo Vision II: Dense Stereo Matching
Stereo Vision II: Dense Stereo Matching Nassir Navab Slides prepared by Christian Unger Outline. Hardware. Challenges. Taxonomy of Stereo Matching. Analysis of Different Problems. Practical Considerations.
More informationCS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric
More informationLecture 14: Computer Vision
CS/b: Artificial Intelligence II Prof. Olga Veksler Lecture : Computer Vision D shape from Images Stereo Reconstruction Many Slides are from Steve Seitz (UW), S. Narasimhan Outline Cues for D shape perception
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More informationComputer Vision I. Dense Stereo Correspondences. Anita Sellent 1/15/16
Computer Vision I Dense Stereo Correspondences Anita Sellent Stereo Two Cameras Overlapping field of view Known transformation between cameras From disparity compute depth [ Bradski, Kaehler: Learning
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More informationCS5670: Computer Vision
CS5670: Computer Vision Noah Snavely Multi-view stereo Announcements Project 3 ( Autostitch ) due Monday 4/17 by 11:59pm Recommended Reading Szeliski Chapter 11.6 Multi-View Stereo: A Tutorial Furukawa
More informationProject 2 due today Project 3 out today. Readings Szeliski, Chapter 10 (through 10.5)
Announcements Stereo Project 2 due today Project 3 out today Single image stereogram, by Niklas Een Readings Szeliski, Chapter 10 (through 10.5) Public Library, Stereoscopic Looking Room, Chicago, by Phillips,
More information3D Photography: Stereo Matching
3D Photography: Stereo Matching Kevin Köser, Marc Pollefeys Spring 2012 http://cvg.ethz.ch/teaching/2012spring/3dphoto/ Stereo & Multi-View Stereo Tsukuba dataset http://cat.middlebury.edu/stereo/ Stereo
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More information3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller
3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationMiniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
More informationStereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33
Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationMulti-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationA virtual tour of free viewpoint rendering
A virtual tour of free viewpoint rendering Cédric Verleysen ICTEAM institute, Université catholique de Louvain, Belgium cedric.verleysen@uclouvain.be Organization of the presentation Context Acquisition
More informationKinectFusion: Real-Time Dense Surface Mapping and Tracking
KinectFusion: Real-Time Dense Surface Mapping and Tracking Gabriele Bleser Thanks to Richard Newcombe for providing the ISMAR slides Overview General: scientific papers (structure, category) KinectFusion:
More informationPerception II: Pinhole camera and Stereo Vision
Perception II: Pinhole camera and Stereo Vision Davide Scaramuzza Margarita Chli, Paul Furgale, Marco Hutter, Roland Siegwart 1 Mobile Robot Control Scheme knowledge, data base mission commands Localization
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationImage processing and features
Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry
More informationBinocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?
Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo
More informationAnnouncements. Stereo Vision Wrapup & Intro Recognition
Announcements Stereo Vision Wrapup & Intro Introduction to Computer Vision CSE 152 Lecture 17 HW3 due date postpone to Thursday HW4 to posted by Thursday, due next Friday. Order of material we ll first
More informationProject Updates Short lecture Volumetric Modeling +2 papers
Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23
More informationInternational Journal for Research in Applied Science & Engineering Technology (IJRASET) A Review: 3D Image Reconstruction From Multiple Images
A Review: 3D Image Reconstruction From Multiple Images Rahul Dangwal 1, Dr. Sukhwinder Singh 2 1 (ME Student) Department of E.C.E PEC University of TechnologyChandigarh, India-160012 2 (Supervisor)Department
More informationMultiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
More informationFinal Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
More informationTwo-View Geometry (Course 23, Lecture D)
Two-View Geometry (Course 23, Lecture D) Jana Kosecka Department of Computer Science George Mason University http://www.cs.gmu.edu/~kosecka General Formulation Given two views of the scene recover the
More informationLecture'9'&'10:'' Stereo'Vision'
Lecture'9'&'10:'' Stereo'Vision' Dr.'Juan'Carlos'Niebles' Stanford'AI'Lab' ' Professor'FeiAFei'Li' Stanford'Vision'Lab' 1' Dimensionality'ReducIon'Machine'(3D'to'2D)' 3D world 2D image Point of observation
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationDirect Methods in Visual Odometry
Direct Methods in Visual Odometry July 24, 2017 Direct Methods in Visual Odometry July 24, 2017 1 / 47 Motivation for using Visual Odometry Wheel odometry is affected by wheel slip More accurate compared
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationProject 4 Results. Representation. Data. Learning. Zachary, Hung-I, Paul, Emanuel. SIFT and HoG are popular and successful.
Project 4 Results Representation SIFT and HoG are popular and successful. Data Hugely varying results from hard mining. Learning Non-linear classifier usually better. Zachary, Hung-I, Paul, Emanuel Project
More informationProf. Trevor Darrell Lecture 18: Multiview and Photometric Stereo
C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 18: Multiview and Photometric Stereo Today Multiview stereo revisited Shape from large image collections Voxel Coloring Digital
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationStereo matching. Francesco Isgrò. 3D Reconstruction and Stereo p.1/21
Stereo matching Francesco Isgrò 3D Reconstruction and Stereo p.1/21 Structure of a stereo vision system Extract interesting point from each image Determine a set of matching points Compute the fundamental
More informationAnnouncements. Stereo
Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative
More informationAccurate and Dense Wide-Baseline Stereo Matching Using SW-POC
Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC Shuji Sakai, Koichi Ito, Takafumi Aoki Graduate School of Information Sciences, Tohoku University, Sendai, 980 8579, Japan Email: sakai@aoki.ecei.tohoku.ac.jp
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationCEng Computational Vision
CEng 583 - Computational Vision 2011-2012 Spring Week 4 18 th of March, 2011 Today 3D Vision Binocular (Multi-view) cues: Stereopsis Motion Monocular cues Shading Texture Familiar size etc. "God must
More informationStereo imaging ideal geometry
Stereo imaging ideal geometry (X,Y,Z) Z f (x L,y L ) f (x R,y R ) Optical axes are parallel Optical axes separated by baseline, b. Line connecting lens centers is perpendicular to the optical axis, and
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More information3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,
3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationRectification and Distortion Correction
Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification
More information3D Photography: Active Ranging, Structured Light, ICP
3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring 2013 http://cvg.ethz.ch/teaching/2013spring/3dphoto/ Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar
More information