GOPRO CAMERAS MATRIX AND DEPTH MAP IN COMPUTER VISION
|
|
- Meghan Simon
- 5 years ago
- Views:
Transcription
1 Tutors : Mr. Yannick Berthoumieu Mrs. Mireille El Gheche GOPRO CAMERAS MATRIX AND DEPTH MAP IN COMPUTER VISION Delmi Elias Kangou Ngoma Joseph Le Goff Baptiste Naji Mohammed Hamza Maamri Kenza Randriamanga Dimby Marchetti Thibault Renauld Vincent Group 1
2 Contents 1 Abstract 2 2 Problem Statement From disparity to depth From optical flow to depth Additional problems Estimation methods based disparity Computing the disparity Inter-camera rectification Illumination variation and occlusion Achieved results Estimation methods based optical flow Computing the optical flow Computing the depth Achieved results Conclusion 9 1/10
3 1 Abstract The depth information is at the heart of various 3D applications such as autonomous driving, augmented reality or navigation. The depth of an object can be estimated by means of different techniques. Stereo imaging using GoPro cameras was privileged because of its accessibility and cost-efficiency among other reasons. This project is part of a broader one which goal is to achieve a high resolution depth map estimation system using a matrix of 9 GoPro Cameras (3x3-1920x1080). 2 Problem Statement 2.1 From disparity to depth A common method for extracting depth information from intensity images is to acquire a pair of images using two cameras displaced from each other by a known distance. Disparity refers to the difference in location of an object in corresponding images (left and right) as seen by the left and right eye. In a pair of images derived from stereo cameras, the apparent motion in pixels for every point can be measured and an intensity image is made out of the measurements. A disparity map refers to the apparent pixel difference or motion between a pair of stereo images. Figure 1 From disparity to depth 2.2 From optical flow to depth There is a second approach in order to obtain depth information. Instead of computing disparity, the optical flow can be analyzed. The optical flow is a quantity which corresponds to a displacement. In stereo vision (two cameras) the optical flow is equal to the movement measured between two similar pixels in the two different pictures from the same scene. In contrast to the disparity calculation, the camera does not need to be perfectly aligned on a plane. Once the optical flow is calculated, two motions are extracted, the plane motion and especially the parallax motion, which contains the depth information. 2.3 Additional problems When dealing with wide-angle cameras such as the Gopro used, distortions appear. These are called radial distortions and are caused by the high distance between the edge of the camera lens and the optical center. This results in the curvature of straight lines, the further the lines are from the optical center the more curved they are. To solve this problem the internal parameters of the Gopro camera must be estimated. The distortion coefficients are related to parameters, each camera has its own parameters, and they can be estimated using different sights taken with the Gopro and a toolbox in Matlab. Several acquisitions of sights are required to get a satisfying estimation and they have to be taken at different angles and positions. 2/10
4 Figure 2 Example of sight used Following this process the Matlab toolboxes were used again to rectify the image. An example can be seen in the following figure. Figure 3 Correction of the distortion 3 Estimation methods based disparity 3.1 Computing the disparity Detecting conjugate pairs in stereo images can be extremely challenging since for each point in the left image, the corresponding point in the right image has to be found. To determine these two points, one in each image, form a conjugate pair, it is necessary to measure the similarity of the points. Thus, before stereo matching, it is unavoidable to locate features which can be matched. Obviously, the points in a uniform region are not good candidates for matching. The variances computed in one direction using all pixels in a window centered about a point are good measures of the distinctness of the point along the direction. The directional variance is given by : I = [f(x, y) f(x + 1, y)] 2 (1) (x,y) S where S represents the pixels in the window. To estimate disparity of points beween two images, two estimators were used: the first one is the minimum mean square error (MMSE) : disparity(x, y) = argmin [f(x, y) g(x + d, y)] 2 (2) d D (x,y) S where f and g are the two images and D is a range of possible values of disparity. However this method is not chosen due to more accurate results obtained with the method of normalized cross correlation (NCC) : disparity(x, y) = argmin d D 1 n x,y S [f(x, y) f][g(x + d, y) ḡ] σ f σ g (3) 3/10
5 where n is the number of pixels in S, f and g barred are the mean of each subimages and the sigmas their standard deviations. However the map computed by block-matching based on NCC is not as accurate as wished because. That is why after computing a first map an iterative algorithm is used aiming to smooth the map, dividing the error pixel percent by four. Illumination variation is then taken into account. 2 solutions were tested : the first uses histogram transfer to fix illumination before computing map disparity and the second simultaneously computes disparity and illumination variation providing maps of disparity and illumination. After tests, the first method is selected. In multiview-case, three images are considered, the middle image is taken as reference. Considering multiple images at once helps with overcoming some issues such as occlusion, and the disparity map obtained happens to be more reliable. To assess the reliability of a disparity map, a comparison was made between the ground truth disparity map and the resulting one. This comparison is quatified by means of errors per pixel. 3.2 Inter-camera rectification The rectification of cameras is basically the transformation that should be applied to the images taken by the 3 3 cameras matrix in order that every pixel of every image is aligned along the horizontal axis. The basic approach is to rectify image 2 by 2. The approach of Vincent Nozick [4]. is using all images from the the matrix cameras at the same time rather than pairwise image rectification. The multi-view rectification, as Vincent Nozick showed, is the transformation of the images in order for them to belong all to a common image plane. In other terms, the rectification process consists in finding a transformation that can be applied to a group of cameras so that their focal plane become coplanar. The particularity of this method is there is no need to know any intrinsic parameters of the cameras and it only requires the knowledge of matching points, also called key points, between the views. Key points are in fact invariant features from every image that can be reliable to match between different views of an object for example. They are invariant to scale, rotation, additional noise and illumination changes. The algorithms SIFT provide a clear method to detect these key points and with the RANSAC algorithm the match can be implemented. If P i = K i [R i C i ] is the projection matrix of each camera, the purpose of this rectification is to find a transformation allowing all focal image planes to be coplanar. The new projection matrix of each camera after the rectification is P i = K i [R i C i ]. The transformation can be assumed as a homographic mixing first of all, a rotation around the optical center, then an update of the focal length. It can be expressed as: where R i = R i R 1 i H i = K i R i K 1 i (4) is the rotation applied to P i in order to have the same orientations as P i. The purpose is to find K i and R i so that the key points are aligned horizontally and vertically with their matches. In this context, two types of rectification were faced: Horizontal rectification : (H i x i k) y y k = 0 (5) Vertical rectification : (H i x k i ) x x k = 0 (6) y k represents the vertical coordinate of the rectified point k on each view. A non-linear process was chosen by optimizing K i and R i to satisfy the equation of vertical rectification by minimizing the residual error over all the corrected points : e = (H i x i k) y y k. The minimizing process is then simplified by reducing the parameters to just the focal length. Hence, the camera s internal parameter matrix is characterized by: 4/10
6 f i 0 w i 2 0 f i h i w i and h i are respectively the width and the height of the image in the i th view and f i = f i 03α i. f i 0 value of the focal length and α i the ratio between the current focal length and the initial value. is the initial 3.3 Illumination variation and occlusion Illumination can have a significant impact on the appearance of surfaces, as the patterns of shading, specularities and shadows change. For instance, some images of a baby under different lighting conditions are shown in the following figure: Figure 4 Lighting difference between two images In order to estimate the illumination variation between two images, the division between these is operated. The results is shown in the following figure: Figure 5 Illumination variation between two images The occlusion between two stereo images represents the obstructions which the baby hides, that cannot be seen from one image to another. The occlusion is handled with two constraints of the disparity map and epipolar geography: Ordering constraint : It means that the pixel order of the left image along the epipolar line must be the same as that of their correspondents. Uniqueness constraint : Two pixels from one image must have at most one matched pixel from the other image. 5/10
7 3.4 Achieved results Vincent Nozick s algorithm must be applied in order that cameras focal planes are aligned along the horizontal axis. OpenCV on Matlab provides both SIFT and RANSAC algorithm, programs that define key points on a first image and then match them with relative key points on a second image. Once this step completed, the "multi-rectification" algorithm provided by Vincent Nozick computes the transformation to be applied using the matching-points. Using a 3 by 3 matrix of cameras, each horizontal triplet will be considered and their focal plane aligned. Let us consider one of those triplets : Figure 6 Initial triplet First of all, characteristic points on each of the 3 images need to be found and to be matched, this is the goal of SIFT algorithm. RANSAC algorithm is also needed in order to remove possible aberrant points. It works on the keypoints detected by SIFT, RANSAC simply keeps pertinent points, called "inliers", and rejects aberrant points called "outliers". The separation occuring improves the robustness of the key-points matching phase. Figure 7 Matched points It can be seen that the algorithms used were able to determine points of correspondence between the images. Finally the homography, computed using the matching-points, has to be applied to each image. 6/10
8 Figure 8 Rectified points It finally can be seen that corresponding pixels are horizontally aligned. Please refer to the attached articles for further information : Epipolar Rectification for Autostereoscopic Camera Setup [2], Multiple View Image Rectification [4], Camera array image rectification and calibration for stereoscopic and autostereoscopic displays [3]. The illumination variation is corrected through histogram transfer : the histogram of the first image is exported to the second image. The results are shown in the following image: Figure 9 Illumination variation corrected with histogram transfer The constraints have been implemented in the theoretical and estimated disparity maps, hence the results in the following figure: Figure 10 Theoretical, estimated disparity and occlusion maps 7/10
9 4 Estimation methods based optical flow 4.1 Computing the optical flow The optical flow was estimated using Matlab s vision toolbox with OpenCV. Among the functions supplied by this toolbox one can find the pixel blocks that are not at the same position in two images of the same scene. Then the result obtained was used to estimate the quantity of movement which are vectors, Vx and Vy, which are related to the movement on the horizontal axis and the movement on the vertical axis. Figure 11 Two different views from the same scene The following figure is a representation of the optical flow : the bluer the area, the more movement within the images, the red areas are those that did not moved. Figure 12 Optical Flow 4.2 Computing the depth In stereovision the position of the two cameras can bound by a homography. This homography is an addition of a rotation and a translation. The position of any pixel in the first image can be estimate by applying the homography to the corresponding pixel from the second image. The optical flow can be divided into a plane and a parallax motion (Eq.(7)). The depth information is included in the parallax motion. ω : optical flow ω π : plane movement µ : parallax movement ω = µ + ω π (7) The plane motion corresponds to the displacement between identical pixels from the first image and pixels from the second one, once applied the rotational component R of the homography and so can be computed according to Eq.(8) and Eq.(9). Indeed a pure rotation does not provide any parallax. 8/10
10 ω π = p i p ωi (8) p i : a pixel from the first image p i : the same R rotated pixel in the second image p ωi = H p i (9) In order to estimate the absolute depth information directly, an epipole need to be calculated, which is difficult to obtain. However the relative depth of a pixel regarding a known depth can be computed Eq.(10). Providing a reference point the relative depth is much easier to compute. γ i γ 1 = µt i (pwi pw1) (10) µ T 1 (pwi pw1) Eventually, the addition of a second reference point provide an enhanced precision. (11) Z map = γ2 γ 1 γ i γ 2 (11) 4.3 Achieved results Using this method decent estimations of the depth map were obtained as shown in the figure below. However the resolution is not optimal due to approximations used in order to compute our results. Figure 13 Input image and corresponding depth map 5 Conclusion Exploring different methods to estimate depth allowed us to compare the results of each method as well as comparing the pros and cons of these methods. The depth map obtained using optical flow is less accurate than the results of the disparity. However the procedure used to estimate the optical flow is easier to generalize to nine cameras whereas with the disparity method it appears difficult to go further than 3 cameras. References [1] Yann Dumortier André Ducrot. Real-Time Quasi Dense Two-Frame Depth Map For Autonomous Guided Vehicles. In: (2013). DOI: 9/10
11 [2] Fabrice Boutarel and Vincent Nozick. Epipolar Rectification for Autostereoscopic Camera Setup. In: EAM Mechatronics Yokohama, Japan, Nov. 2010, pp [3] Vincent Nozick. Camera array image rectification and calibration for stereoscopic and autostereoscopic displays. In: annals of telecommunications (July 2013), pp [4] Vincent Nozick. Multiple View Image Rectification. In: proc. of IEEE-ISAS 2011, International Symposium on Access Spaces. E-ISBN: , print ISBN: Yokohama, Japan, 17-19, June 2011, pp ISBN: [5] Marc Pollefeys. Exploiting scene constraints. URL: www. cs. unc. edu/ ~marc/ tutorial/ node110.htm. 10/10
Adaptive Zoom Distance Measuring System of Camera Based on the Ranging of Binocular Vision
Adaptive Zoom Distance Measuring System of Camera Based on the Ranging of Binocular Vision Zhiyan Zhang 1, Wei Qian 1, Lei Pan 1 & Yanjun Li 1 1 University of Shanghai for Science and Technology, China
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationChaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationCOMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION
COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION Mr.V.SRINIVASA RAO 1 Prof.A.SATYA KALYAN 2 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING PRASAD V POTLURI SIDDHARTHA
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationMultiple View Image Rectification
GS6-B- 20 st International Symposium on Access Spaces (ISAS), IEEE-ISAS 20 Multiple View Image Rectification Vincent Nozick Gaspard Monge Institute, UMR 8049 Paris-Est Marne-la-Vallee University, France
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationRectification and Distortion Correction
Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationComputational Optical Imaging - Optique Numerique. -- Multiple View Geometry and Stereo --
Computational Optical Imaging - Optique Numerique -- Multiple View Geometry and Stereo -- Winter 2013 Ivo Ihrke with slides by Thorsten Thormaehlen Feature Detection and Matching Wide-Baseline-Matching
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationCHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationCSCI 5980: Assignment #3 Homography
Submission Assignment due: Feb 23 Individual assignment. Write-up submission format: a single PDF up to 3 pages (more than 3 page assignment will be automatically returned.). Code and data. Submission
More informationMiniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
More informationApplication questions. Theoretical questions
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationComputational Optical Imaging - Optique Numerique. -- Single and Multiple View Geometry, Stereo matching --
Computational Optical Imaging - Optique Numerique -- Single and Multiple View Geometry, Stereo matching -- Autumn 2015 Ivo Ihrke with slides by Thorsten Thormaehlen Reminder: Feature Detection and Matching
More informationStereo Image Rectification for Simple Panoramic Image Generation
Stereo Image Rectification for Simple Panoramic Image Generation Yun-Suk Kang and Yo-Sung Ho Gwangju Institute of Science and Technology (GIST) 261 Cheomdan-gwagiro, Buk-gu, Gwangju 500-712 Korea Email:{yunsuk,
More informationProject 3 code & artifact due Tuesday Final project proposals due noon Wed (by ) Readings Szeliski, Chapter 10 (through 10.5)
Announcements Project 3 code & artifact due Tuesday Final project proposals due noon Wed (by email) One-page writeup (from project web page), specifying:» Your team members» Project goals. Be specific.
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision I. Dense Stereo Correspondences. Anita Sellent 1/15/16
Computer Vision I Dense Stereo Correspondences Anita Sellent Stereo Two Cameras Overlapping field of view Known transformation between cameras From disparity compute depth [ Bradski, Kaehler: Learning
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More informationCamera model and multiple view geometry
Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then
More informationObject Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More informationComputer Vision I. Announcements. Random Dot Stereograms. Stereo III. CSE252A Lecture 16
Announcements Stereo III CSE252A Lecture 16 HW1 being returned HW3 assigned and due date extended until 11/27/12 No office hours today No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in
More informationRecap from Previous Lecture
Recap from Previous Lecture Tone Mapping Preserve local contrast or detail at the expense of large scale contrast. Changing the brightness within objects or surfaces unequally leads to halos. We are now
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationFinal project bits and pieces
Final project bits and pieces The project is expected to take four weeks of time for up to four people. At 12 hours per week per person that comes out to: ~192 hours of work for a four person team. Capstone:
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
More informationOverview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers
Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationStereoscopic Vision System for reconstruction of 3D objects
Stereoscopic Vision System for reconstruction of 3D objects Robinson Jimenez-Moreno Professor, Department of Mechatronics Engineering, Nueva Granada Military University, Bogotá, Colombia. Javier O. Pinzón-Arenas
More informationCIS 580, Machine Perception, Spring 2015 Homework 1 Due: :59AM
CIS 580, Machine Perception, Spring 2015 Homework 1 Due: 2015.02.09. 11:59AM Instructions. Submit your answers in PDF form to Canvas. This is an individual assignment. 1 Camera Model, Focal Length and
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationAccurate and Dense Wide-Baseline Stereo Matching Using SW-POC
Accurate and Dense Wide-Baseline Stereo Matching Using SW-POC Shuji Sakai, Koichi Ito, Takafumi Aoki Graduate School of Information Sciences, Tohoku University, Sendai, 980 8579, Japan Email: sakai@aoki.ecei.tohoku.ac.jp
More informationRectification. Dr. Gerhard Roth
Rectification Dr. Gerhard Roth Problem Definition Given a pair of stereo images, the intrinsic parameters of each camera, and the extrinsic parameters of the system, R, and, compute the image transformation
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationMidterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
More informationCS5670: Computer Vision
CS5670: Computer Vision Noah Snavely, Zhengqi Li Stereo Single image stereogram, by Niklas Een Mark Twain at Pool Table", no date, UCR Museum of Photography Stereo Given two images from different viewpoints
More informationThree-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras
Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/ Outline The geometry of active stereo.
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Stereo Vision 2 Inferring 3D from 2D Model based pose estimation single (calibrated) camera > Can
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More information3D Reconstruction of a Hopkins Landmark
3D Reconstruction of a Hopkins Landmark Ayushi Sinha (461), Hau Sze (461), Diane Duros (361) Abstract - This paper outlines a method for 3D reconstruction from two images. Our procedure is based on known
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More information3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,
3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationDepth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
More informationFinal Exam Study Guide CSE/EE 486 Fall 2007
Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.
More informationCS 231A Computer Vision (Winter 2018) Problem Set 3
CS 231A Computer Vision (Winter 2018) Problem Set 3 Due: Feb 28, 2018 (11:59pm) 1 Space Carving (25 points) Dense 3D reconstruction is a difficult problem, as tackling it from the Structure from Motion
More informationImage Rectification (Stereo) (New book: 7.2.1, old book: 11.1)
Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture
More informationA Robust Two Feature Points Based Depth Estimation Method 1)
Vol.31, No.5 ACTA AUTOMATICA SINICA September, 2005 A Robust Two Feature Points Based Depth Estimation Method 1) ZHONG Zhi-Guang YI Jian-Qiang ZHAO Dong-Bin (Laboratory of Complex Systems and Intelligence
More informationLecture 14: Computer Vision
CS/b: Artificial Intelligence II Prof. Olga Veksler Lecture : Computer Vision D shape from Images Stereo Reconstruction Many Slides are from Steve Seitz (UW), S. Narasimhan Outline Cues for D shape perception
More informationFlexible Calibration of a Portable Structured Light System through Surface Plane
Vol. 34, No. 11 ACTA AUTOMATICA SINICA November, 2008 Flexible Calibration of a Portable Structured Light System through Surface Plane GAO Wei 1 WANG Liang 1 HU Zhan-Yi 1 Abstract For a portable structured
More informationMachine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy
1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:
More informationThe SIFT (Scale Invariant Feature
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical
More informationRange Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
More information3D Modeling using multiple images Exam January 2008
3D Modeling using multiple images Exam January 2008 All documents are allowed. Answers should be justified. The different sections below are independant. 1 3D Reconstruction A Robust Approche Consider
More information3D Sensing. 3D Shape from X. Perspective Geometry. Camera Model. Camera Calibration. General Stereo Triangulation.
3D Sensing 3D Shape from X Perspective Geometry Camera Model Camera Calibration General Stereo Triangulation 3D Reconstruction 3D Shape from X shading silhouette texture stereo light striping motion mainly
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More information3D Vision Real Objects, Real Cameras. Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun,
3D Vision Real Objects, Real Cameras Chapter 11 (parts of), 12 (parts of) Computerized Image Analysis MN2 Anders Brun, anders@cb.uu.se 3D Vision! Philisophy! Image formation " The pinhole camera " Projective
More informationCompositing a bird's eye view mosaic
Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition
More informationAugmenting Reality, Naturally:
Augmenting Reality, Naturally: Scene Modelling, Recognition and Tracking with Invariant Image Features by Iryna Gordon in collaboration with David G. Lowe Laboratory for Computational Intelligence Department
More informationVision-based endoscope tracking for 3D ultrasound image-guided surgical navigation [Yang et al. 2014, Comp Med Imaging and Graphics]
Vision-based endoscope tracking for 3D ultrasound image-guided surgical navigation [Yang et al. 2014, Comp Med Imaging and Graphics] Gustavo Sato dos Santos IGI Journal Club 23.10.2014 Motivation Goal:
More informationEXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006,
School of Computer Science and Communication, KTH Danica Kragic EXAM SOLUTIONS Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, 14.00 19.00 Grade table 0-25 U 26-35 3 36-45
More informationBinocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?
Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More informationProject 2 due today Project 3 out today. Readings Szeliski, Chapter 10 (through 10.5)
Announcements Stereo Project 2 due today Project 3 out today Single image stereogram, by Niklas Een Readings Szeliski, Chapter 10 (through 10.5) Public Library, Stereoscopic Looking Room, Chicago, by Phillips,
More informationCV: 3D to 2D mathematics. Perspective transformation; camera calibration; stereo computation; and more
CV: 3D to 2D mathematics Perspective transformation; camera calibration; stereo computation; and more Roadmap of topics n Review perspective transformation n Camera calibration n Stereo methods n Structured
More informationROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW
ROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW Thorsten Thormählen, Hellward Broszio, Ingolf Wassermann thormae@tnt.uni-hannover.de University of Hannover, Information Technology Laboratory,
More informationLUMS Mine Detector Project
LUMS Mine Detector Project Using visual information to control a robot (Hutchinson et al. 1996). Vision may or may not be used in the feedback loop. Visual (image based) features such as points, lines
More informationCSCI 5980/8980: Assignment #4. Fundamental Matrix
Submission CSCI 598/898: Assignment #4 Assignment due: March 23 Individual assignment. Write-up submission format: a single PDF up to 5 pages (more than 5 page assignment will be automatically returned.).
More informationOutline 7/2/201011/6/
Outline Pattern recognition in computer vision Background on the development of SIFT SIFT algorithm and some of its variations Computational considerations (SURF) Potential improvement Summary 01 2 Pattern
More informationPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Teesta suspension bridge-darjeeling, India Mark Twain at Pool Table", no date, UCR Museum of Photography Woman getting eye exam during
More informationCameras and Stereo CSE 455. Linda Shapiro
Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation
More informationDepth Measurement and 3-D Reconstruction of Multilayered Surfaces by Binocular Stereo Vision with Parallel Axis Symmetry Using Fuzzy
Depth Measurement and 3-D Reconstruction of Multilayered Surfaces by Binocular Stereo Vision with Parallel Axis Symmetry Using Fuzzy Sharjeel Anwar, Dr. Shoaib, Taosif Iqbal, Mohammad Saqib Mansoor, Zubair
More informationCS 231A Computer Vision (Winter 2014) Problem Set 3
CS 231A Computer Vision (Winter 2014) Problem Set 3 Due: Feb. 18 th, 2015 (11:59pm) 1 Single Object Recognition Via SIFT (45 points) In his 2004 SIFT paper, David Lowe demonstrates impressive object recognition
More informationStereo vision. Many slides adapted from Steve Seitz
Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is
More informationWhat have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
More informationarxiv: v1 [cs.cv] 18 Sep 2017
Direct Pose Estimation with a Monocular Camera Darius Burschka and Elmar Mair arxiv:1709.05815v1 [cs.cv] 18 Sep 2017 Department of Informatics Technische Universität München, Germany {burschka elmar.mair}@mytum.de
More informationOn-line and Off-line 3D Reconstruction for Crisis Management Applications
On-line and Off-line 3D Reconstruction for Crisis Management Applications Geert De Cubber Royal Military Academy, Department of Mechanical Engineering (MSTA) Av. de la Renaissance 30, 1000 Brussels geert.de.cubber@rma.ac.be
More informationFast Outlier Rejection by Using Parallax-Based Rigidity Constraint for Epipolar Geometry Estimation
Fast Outlier Rejection by Using Parallax-Based Rigidity Constraint for Epipolar Geometry Estimation Engin Tola 1 and A. Aydın Alatan 2 1 Computer Vision Laboratory, Ecóle Polytechnique Fédéral de Lausanne
More informationSUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS
SUMMARY: DISTINCTIVE IMAGE FEATURES FROM SCALE- INVARIANT KEYPOINTS Cognitive Robotics Original: David G. Lowe, 004 Summary: Coen van Leeuwen, s1460919 Abstract: This article presents a method to extract
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationOmni Stereo Vision of Cooperative Mobile Robots
Omni Stereo Vision of Cooperative Mobile Robots Zhigang Zhu*, Jizhong Xiao** *Department of Computer Science **Department of Electrical Engineering The City College of the City University of New York (CUNY)
More information