Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

Size: px
Start display at page:

Download "Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov"

Transcription

1 Structured Light II Johannes Köhler Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

2 Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter systems Geometric calibration Today: Structured Light II Gamma calibration Mesh registration 1/7/2015 Lecture 3D Computer Vision 2

3 Gamma function 1/7/2015 Lecture 3D Computer Vision 3

4 Gamma function Phase shifting heavily relies on the different gray values being transmitted correctly Information A Information B Intensity encoding Intensity decoding Physical transmission 4 1/7/2015 Lecture 3D Computer Vision 4

5 Gamma function Problem: Usually projector and camera distort the values being sent / received Result is a non-linear gamma curve of the system x Axis: Intensity values sent by the projector [0 255] y Axis: Intensity values captured by the camera [0 255] 1/7/2015 Lecture 3D Computer Vision 5

6 Gamma function Non-linearity causes large errors in the reconstructions 1/7/2015 Lecture 3D Computer Vision 6

7 Gamma function Solution: Distort the values being sent by the projector in a way, that once they are received, this happens in a linear manner Find function f best describing the systems response curve Exponential function Sigmoid function Use inverse f^{-1} of f and send f^{-1}(x) instead of x Received value: f(f^{-1}(x)) = x 1/7/2015 Lecture 3D Computer Vision 7

8 Gamma function By linearizing the gamma response, the quality of the reconstructions is highly improved 1/7/2015 Lecture 3D Computer Vision 8

9 Matching data + 1/7/2015 Lecture 3D Computer Vision 9

10 Motivation Usually whole objects should be reconstructed from all sides A structured light system can only reconstruct the parts of the objects which are seen by both the camera and the projector Problem: Complex objects 1/7/2015 Lecture 3D Computer Vision 10

11 Recall: Calibration Extrinsics Rel Intrinsics C Intrinsics P 1/7/2015 Lecture 3D Computer Vision 11

12 Motivation Not all details can be captured in one scan Problem: Each scan has its own coordinate system Question: How to align scans? 1/7/2015 Lecture 3D Computer Vision 12

13 Aligning 3D data If the correct correspondences are known, we can find correct relative rotation/translation 1/7/2015 Lecture 3D Computer Vision 13

14 Aligning 3D data How to find correspondences: User input? Feature detection? Signatures? Alternative: Assume closest point 1/7/2015 Lecture 3D Computer Vision 14

15 Aligning 3D data Iterate to find alignment => Algorithm: Iterative Closest Points (ICP) [Besl & McKay 92] Problem: Converges only if starting position is close enough Closest point is often a bad approximation for the corresponding point 1/7/2015 Lecture 3D Computer Vision 15

16 ICP method Extended formulation of the ICP method 1. Selecting source points (from one or both meshes) 2. Matching to points in the other mesh 3. Weighting the correspondences 4. Rejecting outlier point pairs 5. Assigning an error metric to the current transform 6. Minimizing the error metric w.r.t. transformation 1/7/2015 Lecture 3D Computer Vision 16

17 ICP method 1. Step: Selecting source points Use all points Uniform sampling Random sampling Normal-space sampling Ensure that samples have normals distributed as uniformly as possible Uniform sampling Normal-space sampling 1/7/2015 Lecture 3D Computer Vision 17

18 Random vs. normal-space sampling Random sampling Normal-space sampling Normal-space sampling better for mostly-smooth areas with sparse features 1/7/2015 Lecture 3D Computer Vision 18

19 ICP method 2. Step: Matching Closest point => Expensive For range images we can simply project point [Blais 95] Slightly worse performance per iteration 1/7/2015 Lecture 3D Computer Vision 19

20 ICP method 3. Step: Assigning weights Constant weight Assigning lower weights to pairs with greater point-to-point distance : Weight = 1 Dist( p Dist 1, p2) max Weighting based on compatibility of normals Weight = n 1 n 2 Scanner uncertainty 1/7/2015 Lecture 3D Computer Vision 20

21 ICP method 4. Step: Rejecting outliers Point to point distance higher than a given threshold Pairs containing points on end vertices 1/7/2015 Lecture 3D Computer Vision 21

22 ICP method 4. Step: Rejecting outliers Rejection of pairs that are not consistent with their neighboring pairs [Dorai 98] (p1,q1), (p2,q2) are inconsistent <=> Dist( p1, p2) Dist( q1, q2) > τ q1 q2 p2 p1 1/7/2015 Lecture 3D Computer Vision 22

23 ICP method 5. & 6. Step: Error metric and minimization Minimize e.g. f ([ R t]) = 1 N S N S i= 1 qi [ R t] 1 p i 2 Sequential minimization Minimize 1 and 2, then 2 and 3, => Accumulated error (loop closing problem) 1/7/2015 Lecture 3D Computer Vision 23

24 ICP method 5. & 6. Step: Error metric and minimization Simultaneous minimization Simultaneously minimize the various transformations [R i t i ] Diffusively distribute the alignment error over all overlaps of each range images Large computational cost Difficult convergence, if correspondences are updated 1/7/2015 Lecture 3D Computer Vision 24

25 ICP method 6. Step: Minimization by Pulli s method Assumption: Pair-wise alignment yields best result for each pair Align pairs with each other using standard ICP After pair-wise alignment, correspondences can be kept fixed Subsequent global pose minimization converges quickly and distributes error diffusely 1/7/2015 Lecture 3D Computer Vision 25

26 ICP method How to find good initial position? Try to find and match features in both meshes Closely related to image features and matching but in 3D Therefore: Use neighborhood geometric information Descriptors should be Invariant under transform Local Cheap Improves correspondence searching and matching 1/7/2015 Lecture 3D Computer Vision 26

27 ICP method Features: 0.20 Inherent smoothing Related to mean curvature Robust to noise 1/7/2015 Lecture 3D Computer Vision 27

28 ICP method Feature identification Pick as features points with rare descriptor values Results in few but relatively robust correspondences # occurences Features Descriptor value 1/7/2015 Lecture 3D Computer Vision 28

29 ICP method Feature matching Match features with similar descriptor values Use e.g. RANSAC (see lecture 5: Parameter estimation) to filter out wrong correspondences P Q 1/7/2015 Lecture 3D Computer Vision 29

30 ICP method Use features to roughly align the various reconstructions Then use classical ICP for refinement using the good starting approximation Input: 10 scans Alignment Refined by ICP 1/7/2015 Lecture 3D Computer Vision 30

31 SL reconstruction example 31

32 Is ICP a good choice? The previous approach (pre-calibration + subsequent alignment) is by far the most common one in active scanning For alignment, ICP is heavily dominating Does it always yield good results? We will evaluate the global alignment approach of Pulli: Input: N partial scans Pairwise alignment of scans with largest mutual overlapping Global optimization with fixed correspondences 1/7/2015 Lecture 3D Computer Vision 32

33 Synthetic Data Generation 10 meshes are rendered from 8 views each in order to generate a point cloud This data serves as ground truth 1 out of 8 poses is the world, i.e. 70 poses must be estimated To each point cloud besides the world, we apply a small error E = [R t]. This yields fair starting values within ICPs convergence range 1/7/2015 Lecture 3D Computer Vision 33

34 Error computation We apply Pulli s global alignment method to the transformed point clouds; It estimates E E -1 E E = I I is the estimation error. We apply it to every vertex v of the respective pose to obtain a one-dimensional error e = v-i v. 1/7/2015 Lecture 3D Computer Vision 34

35 Error evaluation When is a registration good? Scanner sampling density: Zoom x10 Zoom x100 1/7/2015 Lecture 3D Computer Vision 35

36 Error evaluation Sampling density: Distance between vertices of neighboring pixels is the maximal theoretical precision (for given depth) Note, that this does not regard depth error We consider a registration as good, if the error it introduces is <= the scanner s sampling density, i.e. the registration error is not larger than the theoretical precision at a given depth 1/7/2015 Lecture 3D Computer Vision 36

37 ICP Error distribution 70 estimated poses are evaluated Estimation error propagated to vertices Histogram with error entries (x: Error, y: amount of points; red line is sampling density, note the peak at the right border): 1/7/2015 Lecture 3D Computer Vision 37

38 ICP Error distribution Why are so many points below the sampling density threshold? Peak (failure) for (partially) symmetric objects Weak precision for smooth objects Good precision for objects with complex geometry 1/7/

39 ICP Drawbacks Failure for symmetric objects Weak precision for symmetric objects Strong dependence on sampling strategy 1/7/2015 Lecture 3D Computer Vision 39

40 Registration from Optical Features Structured light scanner is equipped with camera Use of multiple cameras possible and reasonable (no gamma calibration, if equal camera types) Regular camera images allow feature matching 1/7/2015 Lecture 3D Computer Vision 40

41 Registration from Optical Features Regular camera images are perfectly registered to the point cloud/depth map 2D-2D Matches between cameras of different scanner positions thus induce 3D-3D matches Pose estimation from 3D-3D matches: 1/7/2015 Lecture 3D Computer Vision 41

42 Discussion Constant correspondences fast and accurate Independent of geometry Decreased precision for sparse features Failure for symmetric or repetitive features In the literature usually used for initialization of ICP Some (few) authors point out better robustness than ICP Further (potential) drawback: pre-calibration Parameters are kept constant, but might change due to e.g. heat generation, cheap hardware, scanner motion or external influences Distorted geometry, bad fit (see Furuakwa) 1/7/2015 Lecture 3D Computer Vision 42

43 Registration Using Structure from Motion Structure from Motion (SfM) is widely used for estimating camera and structure parameters (see lectures 6/7) Choose camera pair Initialize Intrinsics and Extrinsics Triangulate points Optimize Find most 2D3D-correspondences Compute new Intrinsics and Extrinsics K 0 [ R t] 0 K 1 [ R t] 1 K 2 [ R t] 2 1/7/2015 Lecture 3D Computer Vision 43

44 Registration Using Structure from Motion SfM input (lecture 6/7): 2D correspondences, intrinsics; poses and structure remain to be computed SfM approach (lecture 6/7): Initial pose from 2D-2D correspondences (E-matrix), subsequent poses from 2D-3D correspondences (see lectures 2,6 and 7) During the whole SfM process, it is possible to compute the intrinsics as well (autocalibration). This is beyond the scope of this lecture With autocalibration, only 2D-2D correspondences are required easy projector calibration Consequence: Global scanner calibration by SfM Difference: 2 types of correspondences: camera to projector (phase), camera to camera (e.g. SIFT) 1/7/2015 Lecture 3D Computer Vision 44

45 Registration Using Structure from Motion Instead of [I 0] and relative projector pose [R t], each camera/projector gets a global pose before reconstruction Subsequent reconstruction in common frame 1/7/

46 Relationship to ICP Assume scanner frame given by projector frame (wlog), scanner pose is P SfM computes scanner poses P i in global frame, P i maps from world to local frame Pre-calibration: Geometry computed in local scanner frames, ICP computes mapping to common world. ICP thus computes P -1 1/7/2015 Lecture 3D Computer Vision 46

47 Discussion SfM enables self-calibrating scanner Metric calibration requires special attention More degrees of freedom: Individual parameter set for each device Global optimization easy Use of multiple cameras easily possible, e.g. scanning unit with 2 cameras, 1 projector Projector calibration not necessary 1/7/2015 Lecture 3D Computer Vision 47

48 Evaluation Evaluation with the same dataset Correspondence generation between cameras and projector: Artificial (noisy) phase for camera-projector, texture rendering for camera-camera Task for SfM: Estimate original camera poses used for rendering 1/7/2015 Lecture 3D Computer Vision 48

49 Error distribution For full evaluation, refer to our paper Structure from Motion in the Context of Active Scanning ICP SfM (sharp texture) SfM (blurred texture) SfM (artificial texture) 1/7/2015 Lecture 3D Computer Vision 49

50 Independence from Natural Features 1/7/2015 Lecture 3D Computer Vision 50

51 Conclusion ICP can always be used and must be used if no imagery is available ICP in general performs worse than optical features ICP failure cases: (partially) symmetric geometry Decreased performance for smooth geometry Optical feature failure cases Low quality/ sparse/ symmetric features 1/7/2015 Lecture 3D Computer Vision 51

52 References Besl, McKay A Method for Registration of 3D Shapes; in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, 1992 Rusinkiewicz, Levoy Efficient Variants of the ICP Algorithm, In Proceedings of the Third Intl. Conf. on 3D Digital Imaging and Modeling (2001), pp Dorai, Weng, Jain Registration and Integration of Multiple Object Views for 3D Model Construction, Trans. PAMI, Vol. 20, No. 1, 1998 Gelfand, Mitra, Guibas, Pottmann Robust global registration, in Proceedings of 3 rd Eurographics Symposium on Geometry Processing, /7/2015 Lecture 3D Computer Vision 52

53 References Pulli Multiview registration for large data sets, 3-D Digital Imaging and Modeling, 1999 Furuakwa Multi-view reconstruction for projector camera systems based on bundle adjustment, CVPR workshops, /7/2015 Lecture 3D Computer Vision 53

3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University.

3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University. 3D Computer Vision Structured Light II Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction

More information

Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter

More information

Surface Registration. Gianpaolo Palma

Surface Registration. Gianpaolo Palma Surface Registration Gianpaolo Palma The problem 3D scanning generates multiple range images Each contain 3D points for different parts of the model in the local coordinates of the scanner Find a rigid

More information

5.2 Surface Registration

5.2 Surface Registration Spring 2018 CSCI 621: Digital Geometry Processing 5.2 Surface Registration Hao Li http://cs621.hao-li.com 1 Acknowledgement Images and Slides are courtesy of Prof. Szymon Rusinkiewicz, Princeton University

More information

3D Photography: Active Ranging, Structured Light, ICP

3D Photography: Active Ranging, Structured Light, ICP 3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring 2013 http://cvg.ethz.ch/teaching/2013spring/3dphoto/ Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar

More information

Correspondence. CS 468 Geometry Processing Algorithms. Maks Ovsjanikov

Correspondence. CS 468 Geometry Processing Algorithms. Maks Ovsjanikov Shape Matching & Correspondence CS 468 Geometry Processing Algorithms Maks Ovsjanikov Wednesday, October 27 th 2010 Overall Goal Given two shapes, find correspondences between them. Overall Goal Given

More information

3D Photography: Stereo

3D Photography: Stereo 3D Photography: Stereo Marc Pollefeys, Torsten Sattler Spring 2016 http://www.cvg.ethz.ch/teaching/3dvision/ 3D Modeling with Depth Sensors Today s class Obtaining depth maps / range images unstructured

More information

arxiv: v1 [cs.cv] 28 Sep 2018

arxiv: v1 [cs.cv] 28 Sep 2018 Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,

More information

Introduction to Mobile Robotics Iterative Closest Point Algorithm. Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras

Introduction to Mobile Robotics Iterative Closest Point Algorithm. Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras Introduction to Mobile Robotics Iterative Closest Point Algorithm Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras 1 Motivation 2 The Problem Given: two corresponding point sets: Wanted: translation

More information

Accurate 3D Face and Body Modeling from a Single Fixed Kinect

Accurate 3D Face and Body Modeling from a Single Fixed Kinect Accurate 3D Face and Body Modeling from a Single Fixed Kinect Ruizhe Wang*, Matthias Hernandez*, Jongmoo Choi, Gérard Medioni Computer Vision Lab, IRIS University of Southern California Abstract In this

More information

#$ % $ $& "$%% " $ '$ " '

#$ % $ $& $%%  $ '$  ' ! " This section of the course covers techniques for pairwise (i.e., scanto-scan) and global (i.e., involving more than 2 scans) alignment, given that the algorithms are constrained to obtain a rigid-body

More information

Registration of Dynamic Range Images

Registration of Dynamic Range Images Registration of Dynamic Range Images Tan-Chi Ho 1,2 Jung-Hong Chuang 1 Wen-Wei Lin 2 Song-Sun Lin 2 1 Department of Computer Science National Chiao-Tung University 2 Department of Applied Mathematics National

More information

Rigid ICP registration with Kinect

Rigid ICP registration with Kinect Rigid ICP registration with Kinect Students: Yoni Choukroun, Elie Semmel Advisor: Yonathan Aflalo 1 Overview.p.3 Development of the project..p.3 Papers p.4 Project algorithm..p.6 Result of the whole body.p.7

More information

CRF Based Point Cloud Segmentation Jonathan Nation

CRF Based Point Cloud Segmentation Jonathan Nation CRF Based Point Cloud Segmentation Jonathan Nation jsnation@stanford.edu 1. INTRODUCTION The goal of the project is to use the recently proposed fully connected conditional random field (CRF) model to

More information

A Systems View of Large- Scale 3D Reconstruction

A Systems View of Large- Scale 3D Reconstruction Lecture 23: A Systems View of Large- Scale 3D Reconstruction Visual Computing Systems Goals and motivation Construct a detailed 3D model of the world from unstructured photographs (e.g., Flickr, Facebook)

More information

Multi-view reconstruction for projector camera systems based on bundle adjustment

Multi-view reconstruction for projector camera systems based on bundle adjustment Multi-view reconstruction for projector camera systems based on bundle adjustment Ryo Furuakwa, Faculty of Information Sciences, Hiroshima City Univ., Japan, ryo-f@hiroshima-cu.ac.jp Kenji Inose, Hiroshi

More information

3D Models from Range Sensors. Gianpaolo Palma

3D Models from Range Sensors. Gianpaolo Palma 3D Models from Range Sensors Gianpaolo Palma Who Gianpaolo Palma Researcher at Visual Computing Laboratory (ISTI-CNR) Expertise: 3D scanning, Mesh Processing, Computer Graphics E-mail: gianpaolo.palma@isti.cnr.it

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

3D Scanning. Qixing Huang Feb. 9 th Slide Credit: Yasutaka Furukawa

3D Scanning. Qixing Huang Feb. 9 th Slide Credit: Yasutaka Furukawa 3D Scanning Qixing Huang Feb. 9 th 2017 Slide Credit: Yasutaka Furukawa Geometry Reconstruction Pipeline This Lecture Depth Sensing ICP for Pair-wise Alignment Next Lecture Global Alignment Pairwise Multiple

More information

arxiv: v1 [cs.cv] 28 Sep 2018

arxiv: v1 [cs.cv] 28 Sep 2018 Extrinsic camera calibration method and its performance evaluation Jacek Komorowski 1 and Przemyslaw Rokita 2 arxiv:1809.11073v1 [cs.cv] 28 Sep 2018 1 Maria Curie Sklodowska University Lublin, Poland jacek.komorowski@gmail.com

More information

Project Updates Short lecture Volumetric Modeling +2 papers

Project Updates Short lecture Volumetric Modeling +2 papers Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23

More information

Structured Light. Tobias Nöll Thanks to Marc Pollefeys, David Nister and David Lowe

Structured Light. Tobias Nöll Thanks to Marc Pollefeys, David Nister and David Lowe Structured Light Tobias Nöll tobias.noell@dfki.de Thanks to Marc Pollefeys, David Nister and David Lowe Introduction Previous lecture: Dense reconstruction Dense matching of non-feature pixels Patch-based

More information

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller

3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller 3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

Algorithm research of 3D point cloud registration based on iterative closest point 1

Algorithm research of 3D point cloud registration based on iterative closest point 1 Acta Technica 62, No. 3B/2017, 189 196 c 2017 Institute of Thermomechanics CAS, v.v.i. Algorithm research of 3D point cloud registration based on iterative closest point 1 Qian Gao 2, Yujian Wang 2,3,

More information

Structured light 3D reconstruction

Structured light 3D reconstruction Structured light 3D reconstruction Reconstruction pipeline and industrial applications rodola@dsi.unive.it 11/05/2010 3D Reconstruction 3D reconstruction is the process of capturing the shape and appearance

More information

Overview. Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion

Overview. Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion Overview Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion Binary-Space-Partitioned Images 3-D Surface Extraction from Medical

More information

FAST REGISTRATION OF TERRESTRIAL LIDAR POINT CLOUD AND SEQUENCE IMAGES

FAST REGISTRATION OF TERRESTRIAL LIDAR POINT CLOUD AND SEQUENCE IMAGES FAST REGISTRATION OF TERRESTRIAL LIDAR POINT CLOUD AND SEQUENCE IMAGES Jie Shao a, Wuming Zhang a, Yaqiao Zhu b, Aojie Shen a a State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction

More information

3D object recognition used by team robotto

3D object recognition used by team robotto 3D object recognition used by team robotto Workshop Juliane Hoebel February 1, 2016 Faculty of Computer Science, Otto-von-Guericke University Magdeburg Content 1. Introduction 2. Depth sensor 3. 3D object

More information

Algorithms for 3D Isometric Shape Correspondence

Algorithms for 3D Isometric Shape Correspondence Algorithms for 3D Isometric Shape Correspondence Yusuf Sahillioğlu Computer Eng. Dept., Koç University, Istanbul, Turkey (PhD) Computer Eng. Dept., METU, Ankara, Turkey (Asst. Prof.) 2 / 53 Problem Definition

More information

Improving Initial Estimations for Structure from Motion Methods

Improving Initial Estimations for Structure from Motion Methods Improving Initial Estimations for Structure from Motion Methods University of Bonn Outline Motivation Computer-Vision Basics Stereo Vision Bundle Adjustment Feature Matching Global Initial Estimation Component

More information

3D Perception. CS 4495 Computer Vision K. Hawkins. CS 4495 Computer Vision. 3D Perception. Kelsey Hawkins Robotics

3D Perception. CS 4495 Computer Vision K. Hawkins. CS 4495 Computer Vision. 3D Perception. Kelsey Hawkins Robotics CS 4495 Computer Vision Kelsey Hawkins Robotics Motivation What do animals, people, and robots want to do with vision? Detect and recognize objects/landmarks Find location of objects with respect to themselves

More information

Fast and robust techniques for 3D/2D registration and photo blending on massive point clouds

Fast and robust techniques for 3D/2D registration and photo blending on massive point clouds www.crs4.it/vic/ vcg.isti.cnr.it/ Fast and robust techniques for 3D/2D registration and photo blending on massive point clouds R. Pintus, E. Gobbetti, M.Agus, R. Combet CRS4 Visual Computing M. Callieri

More information

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10

Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10 Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley

More information

Processing 3D Surface Data

Processing 3D Surface Data Processing 3D Surface Data Computer Animation and Visualisation Lecture 12 Institute for Perception, Action & Behaviour School of Informatics 3D Surfaces 1 3D surface data... where from? Iso-surfacing

More information

Segmentation and Tracking of Partial Planar Templates

Segmentation and Tracking of Partial Planar Templates Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract

More information

Scan Matching. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics

Scan Matching. Pieter Abbeel UC Berkeley EECS. Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Scan Matching Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Scan Matching Overview Problem statement: Given a scan and a map, or a scan and a scan,

More information

Feature Matching and Robust Fitting

Feature Matching and Robust Fitting Feature Matching and Robust Fitting Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Project 2 questions? This

More information

Geometric Modeling and Processing

Geometric Modeling and Processing Geometric Modeling and Processing Tutorial of 3DIM&PVT 2011 (Hangzhou, China) May 16, 2011 4. Geometric Registration 4.1 Rigid Registration Range Scanning: Reconstruction Set of raw scans Reconstructed

More information

Image processing and features

Image processing and features Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry

More information

Dynamic Geometry Processing

Dynamic Geometry Processing Dynamic Geometry Processing EG 2012 Tutorial Will Chang, Hao Li, Niloy Mitra, Mark Pauly, Michael Wand Tutorial: Dynamic Geometry Processing 1 Articulated Global Registration Introduction and Overview

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric

More information

3D Computer Vision. Structured Light I. Prof. Didier Stricker. Kaiserlautern University.

3D Computer Vision. Structured Light I. Prof. Didier Stricker. Kaiserlautern University. 3D Computer Vision Structured Light I Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction

More information

Dynamic Time Warping for Binocular Hand Tracking and Reconstruction

Dynamic Time Warping for Binocular Hand Tracking and Reconstruction Dynamic Time Warping for Binocular Hand Tracking and Reconstruction Javier Romero, Danica Kragic Ville Kyrki Antonis Argyros CAS-CVAP-CSC Dept. of Information Technology Institute of Computer Science KTH,

More information

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking. In close-up photo, the depth of field is limited. Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

More information

Lecture 10: Multi view geometry

Lecture 10: Multi view geometry Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from

More information

Robust Range Image Registration using a Common Plane

Robust Range Image Registration using a Common Plane VRVis Technical Report 1 Robust Range Image Registration using a Common Plane Joachim Bauer bauer@icg.vrvis.at Konrad Karner karner@vrvis.at Andreas Klaus klaus@vrvis.at Roland Perko University of Technology

More information

A 3D Point Cloud Registration Algorithm based on Feature Points

A 3D Point Cloud Registration Algorithm based on Feature Points International Conference on Information Sciences, Machinery, Materials and Energy (ICISMME 2015) A 3D Point Cloud Registration Algorithm based on Feature Points Yi Ren 1, 2, a, Fucai Zhou 1, b 1 School

More information

Incremental Structured ICP Algorithm

Incremental Structured ICP Algorithm Incremental Structured ICP Algorithm Haokun Geng, Johnny Chien, Radu Nicolescu, and Reinhard Klette The.enpeda.. Project, Tamaki Campus The University of Auckland, New Zealand Abstract. Variants of the

More information

Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013

Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013 Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth

More information

Stochastic Optimization for Rigid Point Set Registration

Stochastic Optimization for Rigid Point Set Registration Stochastic Optimization for Rigid Point Set Registration Chavdar Papazov and Darius Burschka Machine Vision and Perception Group (MVP) Department of Computer Science Technische Universität München, Germany

More information

A Multi-Resolution ICP with Heuristic Closest Point Search for Fast and Robust 3D Registration of Range Images

A Multi-Resolution ICP with Heuristic Closest Point Search for Fast and Robust 3D Registration of Range Images A Multi-Resolution ICP with Heuristic Closest Point Search for Fast and Robust 3D Registration of Range Images Timothée Jost and Heinz Hügli Institute of Microtechnology, University of Neuchâtel, Breguet

More information

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Zhen Qin (University of California, Riverside) Peter van Beek & Xu Chen (SHARP Labs of America, Camas, WA) 2015/8/30

More information

Perceptual Grouping from Motion Cues Using Tensor Voting

Perceptual Grouping from Motion Cues Using Tensor Voting Perceptual Grouping from Motion Cues Using Tensor Voting 1. Research Team Project Leader: Graduate Students: Prof. Gérard Medioni, Computer Science Mircea Nicolescu, Changki Min 2. Statement of Project

More information

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1

More information

Estimation of Camera Pose with Respect to Terrestrial LiDAR Data

Estimation of Camera Pose with Respect to Terrestrial LiDAR Data Estimation of Camera Pose with Respect to Terrestrial LiDAR Data Wei Guan Suya You Guan Pang Computer Science Department University of Southern California, Los Angeles, USA Abstract In this paper, we present

More information

Multiview Stereo COSC450. Lecture 8

Multiview Stereo COSC450. Lecture 8 Multiview Stereo COSC450 Lecture 8 Stereo Vision So Far Stereo and epipolar geometry Fundamental matrix captures geometry 8-point algorithm Essential matrix with calibrated cameras 5-point algorithm Intersect

More information

Translation Symmetry Detection: A Repetitive Pattern Analysis Approach

Translation Symmetry Detection: A Repetitive Pattern Analysis Approach 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops Translation Symmetry Detection: A Repetitive Pattern Analysis Approach Yunliang Cai and George Baciu GAMA Lab, Department of Computing

More information

Prof. Fanny Ficuciello Robotics for Bioengineering Visual Servoing

Prof. Fanny Ficuciello Robotics for Bioengineering Visual Servoing Visual servoing vision allows a robotic system to obtain geometrical and qualitative information on the surrounding environment high level control motion planning (look-and-move visual grasping) low level

More information

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure

More information

Large Scale 3D Reconstruction by Structure from Motion

Large Scale 3D Reconstruction by Structure from Motion Large Scale 3D Reconstruction by Structure from Motion Devin Guillory Ziang Xie CS 331B 7 October 2013 Overview Rome wasn t built in a day Overview of SfM Building Rome in a Day Building Rome on a Cloudless

More information

Feature Based Registration - Image Alignment

Feature Based Registration - Image Alignment Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html

More information

STRUCTURAL ICP ALGORITHM FOR POSE ESTIMATION BASED ON LOCAL FEATURES

STRUCTURAL ICP ALGORITHM FOR POSE ESTIMATION BASED ON LOCAL FEATURES STRUCTURAL ICP ALGORITHM FOR POSE ESTIMATION BASED ON LOCAL FEATURES Marco A. Chavarria, Gerald Sommer Cognitive Systems Group. Christian-Albrechts-University of Kiel, D-2498 Kiel, Germany {mc,gs}@ks.informatik.uni-kiel.de

More information

CS443: Digital Imaging and Multimedia Binary Image Analysis. Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University

CS443: Digital Imaging and Multimedia Binary Image Analysis. Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University CS443: Digital Imaging and Multimedia Binary Image Analysis Spring 2008 Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines A Simple Machine Vision System Image segmentation by thresholding

More information

Geometric Registration for Deformable Shapes 1.1 Introduction

Geometric Registration for Deformable Shapes 1.1 Introduction Geometric Registration for Deformable Shapes 1.1 Introduction Overview Data Sources and Applications Problem Statement Overview Presenters Will Chang University of California at San Diego, USA Hao Li ETH

More information

Surface Reconstruction. Gianpaolo Palma

Surface Reconstruction. Gianpaolo Palma Surface Reconstruction Gianpaolo Palma Surface reconstruction Input Point cloud With or without normals Examples: multi-view stereo, union of range scan vertices Range scans Each scan is a triangular mesh

More information

REFINEMENT OF COLORED MOBILE MAPPING DATA USING INTENSITY IMAGES

REFINEMENT OF COLORED MOBILE MAPPING DATA USING INTENSITY IMAGES REFINEMENT OF COLORED MOBILE MAPPING DATA USING INTENSITY IMAGES T. Yamakawa a, K. Fukano a,r. Onodera a, H. Masuda a, * a Dept. of Mechanical Engineering and Intelligent Systems, The University of Electro-Communications,

More information

CS 468 Data-driven Shape Analysis. Shape Descriptors

CS 468 Data-driven Shape Analysis. Shape Descriptors CS 468 Data-driven Shape Analysis Shape Descriptors April 1, 2014 What Is A Shape Descriptor? Shapes Shape Descriptor F1=[f1, f2,.., fn] F2=[f1, f2,.., fn] F3=[f1, f2,.., fn] What Is A Shape Descriptor?

More information

What is Computer Vision?

What is Computer Vision? Perceptual Grouping in Computer Vision Gérard Medioni University of Southern California What is Computer Vision? Computer Vision Attempt to emulate Human Visual System Perceive visual stimuli with cameras

More information

Robotics Programming Laboratory

Robotics Programming Laboratory Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car

More information

Estimation of common groundplane based on co-motion statistics

Estimation of common groundplane based on co-motion statistics Estimation of common groundplane based on co-motion statistics Zoltan Szlavik, Laszlo Havasi 2, Tamas Sziranyi Analogical and Neural Computing Laboratory, Computer and Automation Research Institute of

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

The correspondence problem. A classic problem. A classic problem. Deformation-Drive Shape Correspondence. Fundamental to geometry processing

The correspondence problem. A classic problem. A classic problem. Deformation-Drive Shape Correspondence. Fundamental to geometry processing The correspondence problem Deformation-Drive Shape Correspondence Hao (Richard) Zhang 1, Alla Sheffer 2, Daniel Cohen-Or 3, Qingnan Zhou 2, Oliver van Kaick 1, and Andrea Tagliasacchi 1 July 3, 2008 1

More information

Hand-eye calibration with a depth camera: 2D or 3D?

Hand-eye calibration with a depth camera: 2D or 3D? Hand-eye calibration with a depth camera: 2D or 3D? Svenja Kahn 1, Dominik Haumann 2 and Volker Willert 2 1 Fraunhofer IGD, Darmstadt, Germany 2 Control theory and robotics lab, TU Darmstadt, Darmstadt,

More information

EECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems

EECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate

More information

Geometric Registration for Deformable Shapes 2.2 Deformable Registration

Geometric Registration for Deformable Shapes 2.2 Deformable Registration Geometric Registration or Deormable Shapes 2.2 Deormable Registration Variational Model Deormable ICP Variational Model What is deormable shape matching? Example? What are the Correspondences? Eurographics

More information

BIL Computer Vision Apr 16, 2014

BIL Computer Vision Apr 16, 2014 BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm

More information

ENGN D Photography / Spring 2018 / SYLLABUS

ENGN D Photography / Spring 2018 / SYLLABUS ENGN 2502 3D Photography / Spring 2018 / SYLLABUS Description of the proposed course Over the last decade digital photography has entered the mainstream with inexpensive, miniaturized cameras routinely

More information

Chaplin, Modern Times, 1936

Chaplin, Modern Times, 1936 Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections

More information

Real-time Image-based Reconstruction of Pipes Using Omnidirectional Cameras

Real-time Image-based Reconstruction of Pipes Using Omnidirectional Cameras Real-time Image-based Reconstruction of Pipes Using Omnidirectional Cameras Dipl. Inf. Sandro Esquivel Prof. Dr.-Ing. Reinhard Koch Multimedia Information Processing Christian-Albrechts-University of Kiel

More information

Multi-View Image Coding in 3-D Space Based on 3-D Reconstruction

Multi-View Image Coding in 3-D Space Based on 3-D Reconstruction Multi-View Image Coding in 3-D Space Based on 3-D Reconstruction Yongying Gao and Hayder Radha Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48823 email:

More information

Iterative Closest Point Algorithm in the Presence of Anisotropic Noise

Iterative Closest Point Algorithm in the Presence of Anisotropic Noise Iterative Closest Point Algorithm in the Presence of Anisotropic Noise L. Maier-Hein, T. R. dos Santos, A. M. Franz, H.-P. Meinzer German Cancer Research Center, Div. of Medical and Biological Informatics

More information

ROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW

ROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW ROBUST LINE-BASED CALIBRATION OF LENS DISTORTION FROM A SINGLE VIEW Thorsten Thormählen, Hellward Broszio, Ingolf Wassermann thormae@tnt.uni-hannover.de University of Hannover, Information Technology Laboratory,

More information

Camera Drones Lecture 3 3D data generation

Camera Drones Lecture 3 3D data generation Camera Drones Lecture 3 3D data generation Ass.Prof. Friedrich Fraundorfer WS 2017 Outline SfM introduction SfM concept Feature matching Camera pose estimation Bundle adjustment Dense matching Data products

More information

CS664 Lecture #16: Image registration, robust statistics, motion

CS664 Lecture #16: Image registration, robust statistics, motion CS664 Lecture #16: Image registration, robust statistics, motion Some material taken from: Alyosha Efros, CMU http://www.cs.cmu.edu/~efros Xenios Papademetris http://noodle.med.yale.edu/~papad/various/papademetris_image_registration.p

More information

EXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006,

EXAM SOLUTIONS. Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, School of Computer Science and Communication, KTH Danica Kragic EXAM SOLUTIONS Image Processing and Computer Vision Course 2D1421 Monday, 13 th of March 2006, 14.00 19.00 Grade table 0-25 U 26-35 3 36-45

More information

Automatic Image Alignment (feature-based)

Automatic Image Alignment (feature-based) Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature

More information

CSE 527: Introduction to Computer Vision

CSE 527: Introduction to Computer Vision CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature

More information

Alignment of Continuous Video onto 3D Point Clouds

Alignment of Continuous Video onto 3D Point Clouds 1 Alignment of Continuous Video onto 3D Point Clouds W. Zhao 1, D. Nister 2, and S. Hsu Sarnoff Corporation 201 Washington Road Princeton, NJ 08540, USA email: { wzhao, dnister, shsu }@sarnoff.com Tel:

More information

Two-view geometry Computer Vision Spring 2018, Lecture 10

Two-view geometry Computer Vision Spring 2018, Lecture 10 Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of

More information

Efficient 3D Shape Acquisition and Registration Using Hybrid Scanning Data

Efficient 3D Shape Acquisition and Registration Using Hybrid Scanning Data Efficient 3D Shape Acquisition and Registration Using Hybrid Scanning Data Hongwei Zheng, Dietmar Saupe, Markus Roth, Andreas Böhler and Peter Opuchlik Computer and Information Science, University of Konstanz,

More information

Partially overlapped range images registration based on fringe pattern profilometry

Partially overlapped range images registration based on fringe pattern profilometry University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2011 Partially overlapped range images registration based on fringe pattern

More information

3D Environment Reconstruction

3D Environment Reconstruction 3D Environment Reconstruction Using Modified Color ICP Algorithm by Fusion of a Camera and a 3D Laser Range Finder The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems October 11-15,

More information

CS233: The Shape of Data Handout # 3 Geometric and Topological Data Analysis Stanford University Wednesday, 9 May 2018

CS233: The Shape of Data Handout # 3 Geometric and Topological Data Analysis Stanford University Wednesday, 9 May 2018 CS233: The Shape of Data Handout # 3 Geometric and Topological Data Analysis Stanford University Wednesday, 9 May 2018 Homework #3 v4: Shape correspondences, shape matching, multi-way alignments. [100

More information

Generation of Triangle Meshes from Time-of-Flight Data for Surface Registration

Generation of Triangle Meshes from Time-of-Flight Data for Surface Registration Generation of Triangle Meshes from Time-of-Flight Data for Surface Registration Thomas Kilgus, Thiago R. dos Santos, Alexander Seitel, Kwong Yung, Alfred M. Franz, Anja Groch, Ivo Wolf, Hans-Peter Meinzer,

More information

Multi-View 3D-Reconstruction

Multi-View 3D-Reconstruction Multi-View 3D-Reconstruction Cedric Cagniart Computer Aided Medical Procedures (CAMP) Technische Universität München, Germany 1 Problem Statement Given several calibrated views of an object... can we automatically

More information

Stereo and Epipolar geometry

Stereo and Epipolar geometry Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka

More information

Computational Optical Imaging - Optique Numerique. -- Multiple View Geometry and Stereo --

Computational Optical Imaging - Optique Numerique. -- Multiple View Geometry and Stereo -- Computational Optical Imaging - Optique Numerique -- Multiple View Geometry and Stereo -- Winter 2013 Ivo Ihrke with slides by Thorsten Thormaehlen Feature Detection and Matching Wide-Baseline-Matching

More information

Stereo Vision. MAN-522 Computer Vision

Stereo Vision. MAN-522 Computer Vision Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in

More information

CSE 527: Introduction to Computer Vision

CSE 527: Introduction to Computer Vision CSE 527: Introduction to Computer Vision Week 10 Class 2: Visual Odometry November 2nd, 2017 Today Visual Odometry Intro Algorithm SLAM Visual Odometry Input Output Images, Video Camera trajectory, motion

More information