Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures
|
|
|
- Suzan McGee
- 6 years ago
- Views:
Transcription
1 Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures Robert Maier, Jörg Stückler, Daniel Cremers International Conference on 3D Vision (3DV) October 2015, Lyon, France
2 Motivation Given: Computer Vision Group Low-resolution RGB-D frames (640 x 480) Accurate geometric reconstruction R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 2
3 Motivation Given: Computer Vision Group Low-resolution RGB-D frames (640 x 480) Accurate geometric State-of-the-art w.r.t. visual appearance: reconstruction Vertex colors: limited resolution Texture mapping: good-quality results, but slow/impractical R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 2
4 Motivation Given: Computer Vision Group Low-resolution RGB-D frames (640 x 480) Accurate geometric State-of-the-art w.r.t. visual appearance: reconstruction Vertex colors: limited resolution Texture mapping: good-quality results, but slow/impractical Problem: Gap in research of fast and robust estimation of highquality visual appearance from low-cost RGB-D sensors R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 2
5 Texture Mapping with Super-Resolution Keyframes Our Approach: R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 3
6 Texture Mapping with Super-Resolution Keyframes Our Approach: Super-resolution (SR) Keyframe Fusion and Deblurring R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 3
7 Texture Mapping with Super-Resolution Keyframes Our Approach: Super-resolution (SR) Keyframe Fusion and Deblurring Texture Mapping using SR keyframes (weighted median) R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 3
8 Texture Mapping with Super-Resolution Keyframes Our Approach: Super-resolution (SR) Keyframe Fusion and Deblurring Texture Mapping using SR keyframes (weighted median) Vertex colors Our approach R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 3
9 Related Work R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 4
10 Related Work Vertex colors (weighted average) Newcombe et al., KinectFusion: Real-time dense surface mapping and tracking. ISMAR 2011 Sturm et al., CopyMe3D: Scanning and printing persons in 3D. GCPR 2013 R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 4
11 Related Work Vertex colors (weighted average) Newcombe et al., KinectFusion: Real-time dense surface mapping and tracking. ISMAR 2011 Sturm et al., CopyMe3D: Scanning and printing persons in 3D. GCPR 2013 Texture Mapping Weighted Median [Coorg and Teller. Automatic extraction of textured vertical facades from pose imagery. 1998] Single input view per face, minimize seams [Gal et al., Seamless montage for texturing models. 2010] Variational super-resolution [Goldlücke et al., A super-resolution framework for high-accuracy multiview reconstruction, IJCV 2014] R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 4
12 Related Work Vertex colors (weighted average) Newcombe et al., KinectFusion: Real-time dense surface mapping and tracking. ISMAR 2011 Sturm et al., CopyMe3D: Scanning and printing persons in 3D. GCPR 2013 Texture Mapping Weighted Median [Coorg and Teller. Automatic extraction of textured vertical facades from pose imagery. 1998] Single input view per face, minimize seams [Gal et al., Seamless montage for texturing models. 2010] Variational super-resolution [Goldlücke et al., A super-resolution framework for high-accuracy multiview reconstruction, IJCV 2014] RGB-D based 3D Reconstruction Super-resolution RGB-D keyframes [Meilland and Comport. Super-resolution 3D tracking and mapping. ICRA 2013] Optimization of camera poses and non-rigid correction [Zhou and Koltun. Color map optimization for 3D reconstruction with consumer depth cameras. TOG 2014] R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 4
13 Related Work Vertex colors (weighted average) Newcombe et al., KinectFusion: Real-time dense surface mapping and tracking. ISMAR 2011 Sturm et al., CopyMe3D: Scanning and printing persons in 3D. GCPR 2013 Texture Mapping Weighted Median [Coorg and Teller. Automatic extraction of textured vertical facades from pose imagery. 1998] Single input view per face, minimize seams [Gal et al., Seamless montage for texturing models. 2010] Variational super-resolution [Goldlücke et al., A super-resolution framework for high-accuracy multiview reconstruction, IJCV 2014] RGB-D based 3D Reconstruction Super-resolution RGB-D keyframes [Meilland and Comport. Super-resolution 3D tracking and mapping. ICRA 2013] Optimization of camera poses and non-rigid correction [Zhou and Koltun. Color map optimization for 3D reconstruction with consumer depth cameras. TOG 2014] Contribution: Texture Mapping using Super-resolution Keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 4
14 System Overview R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 5
15 System Overview Low-resolution RGB-D frames R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 5
16 System Overview Low-resolution RGB-D frames DVO-SLAM TSDF Volume Integration 3D Model R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 5
17 System Overview Low-resolution RGB-D frames DVO-SLAM TSDF Volume Integration 3D Model Keyframe Fusion Super-resolution Keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 5
18 System Overview Low-resolution RGB-D frames DVO-SLAM TSDF Volume Integration 3D Model Parametrization Texture Keyframe Fusion Super-resolution Keyframes Texel Color Computation R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 5
19 Geometric 3D Reconstruction R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 6
20 Geometric 3D Reconstruction DVO-SLAM: camera trajectory [Kerl et al., Dense visual slam for RGB-D cameras. IROS 2013] Real-time 3D reconstruction on a CPU Robust Dense Visual Odometry Loop closure detection + pose graph optimization R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 6
21 Geometric 3D Reconstruction DVO-SLAM: camera trajectory [Kerl et al., Dense visual slam for RGB-D cameras. IROS 2013] Real-time 3D reconstruction on a CPU Robust Dense Visual Odometry Loop closure detection + pose graph optimization Model Fusion (TSDF Volume): 3D mesh R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 6
22 Vertex Color Computation Mesh vertex v, input views C i (blur measure b i ), camera poses R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 7
23 Vertex Color Computation Mesh vertex v, input views C i (blur measure b i ), camera poses Compute observations (x i is obs. of v in view i): x 0 x 1 R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 7
24 Vertex Color Computation Mesh vertex v, input views C i (blur measure b i ), camera poses Compute observations (x i is obs. of v in view i): Discard x i close to depth discontinuities x 0 x 1 R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 7
25 Vertex Color Computation Mesh vertex v, input views C i (blur measure b i ), camera poses Compute observations (x i is obs. of v in view i): Discard x i close to depth discontinuities Weights of obs. x i : x 0 x 1 R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 7
26 Vertex Color Computation Mesh vertex v, input views C i (blur measure b i ), camera poses Compute observations (x i is obs. of v in view i): Discard x i close to depth discontinuities Weights of obs. x i : x 0 Compute vertex color x: Weighted mean: x 1 Weighted median: R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 7
27 Vertex Color Computation Unweighted Mean Weighted Mean Weighted Median R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 8
28 Texture Mapping using SR Keyframes Per-vertex colors: limited resolution Increase resolution: Texture Approach: SR Keyframe Fusion Texture Mapping from SR keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 9
29 Keyframe Fusion Idea: fuse low-resolution (LR) input RGB-D frames into high resolution RGB-D keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 10
30 Keyframe Fusion Idea: fuse low-resolution (LR) input RGB-D frames into high resolution RGB-D keyframes Depth fusion Warp LR depth maps into keyframe (using relative poses) Upsample and fuse depth using weighted averaging R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 10
31 Keyframe Fusion Idea: fuse low-resolution (LR) input RGB-D frames into high resolution RGB-D keyframes Depth fusion Warp LR depth maps into keyframe (using relative poses) Upsample and fuse depth using weighted averaging Color fusion Deconvolution: Wiener Filter on LR input images Warp fused keyframe depth to input images for color lookup Fuse colors using weighted median R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 10
32 Keyframe Fusion LR input image Fused SR keyframe R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 11
33 Texture Mapping using SR Keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 12
34 Texture Mapping using SR Keyframes Texture Parametrization: One-to-one mapping between 3D mesh and 2D texture R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 12
35 Texture Mapping using SR Keyframes Texture Parametrization: One-to-one mapping between 3D mesh and 2D texture Texel color computation: Compute 3D vertex for 2D texel (based on enclosing triangle using barycentric coordinates) Compute color from SR keyframes analogous to per-vertex recoloring scheme (weighted median) R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 12
36 Qualitative Evaluation: Deconvolution Without deconvolution With deconvolution R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 13
37 Qualitative Evaluation: SR Keyframe resolution Keyframe dimensions 1280 x 960 (scale s = 2) Keyframe dimensions 2560 x 1920 (scale s = 4) R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 14
38 Qualitative Evaluation: LR input frames vs. SR keyframes With LR input frames With SR keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 15
39 Runtime Evaluation Datasets: R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 16
40 Runtime Evaluation Datasets: Runtimes: (Standard desktop PC with Intel Core i CPU with 3.40GHz and 8GB RAM) R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 16
41 Phone dataset RGB input images Vertex colors Our approach R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 17
42 Keyboard dataset Vertex colors Our approach RGB input images R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 18
43 Conclusion Computer Vision Group Robust and efficient method for high-quality texture mapping in RGB-D-based 3D reconstruction R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 19
44 Conclusion Computer Vision Group Robust and efficient method for high-quality texture mapping in RGB-D-based 3D reconstruction Fuse low-quality color images into SR keyframes R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 19
45 Conclusion Computer Vision Group Robust and efficient method for high-quality texture mapping in RGB-D-based 3D reconstruction Fuse low-quality color images into SR keyframes Map high-quality keyframes onto 3D model texture using weighted median scheme R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 19
46 Conclusion Computer Vision Group Robust and efficient method for high-quality texture mapping in RGB-D-based 3D reconstruction Fuse low-quality color images into SR keyframes Map high-quality keyframes onto 3D model texture using weighted median scheme Experimental results: Increased photo-realism of reconstructed 3D models Very efficient and practical post-processing step (runtimes within a few minutes) R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 19
47 Conclusion Computer Vision Group Robust and efficient method for high-quality texture mapping in RGB-D-based 3D reconstruction Fuse low-quality color images into SR keyframes Map high-quality keyframes onto 3D model texture using weighted median scheme Experimental results: Increased photo-realism of reconstructed 3D models Very efficient and practical post-processing step (runtimes within a few minutes) Thank you! R. Maier, J. Stückler, D. Cremers: Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures 19
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction
MAIER ET AL.: EFFICIENT LARGE-SCALE ONLINE SURFACE CORRECTION 1 Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction Robert Maier maierr@in.tum.de Raphael Schaller schaller@in.tum.de
CVPR 2014 Visual SLAM Tutorial Kintinuous
CVPR 2014 Visual SLAM Tutorial Kintinuous kaess@cmu.edu The Robotics Institute Carnegie Mellon University Recap: KinectFusion [Newcombe et al., ISMAR 2011] RGB-D camera GPU 3D/color model RGB TSDF (volumetric
Scanning and Printing Objects in 3D Jürgen Sturm
Scanning and Printing Objects in 3D Jürgen Sturm Metaio (formerly Technical University of Munich) My Research Areas Visual navigation for mobile robots RoboCup Kinematic Learning Articulated Objects Quadrocopters
Dense 3D Reconstruction from Autonomous Quadrocopters
Dense 3D Reconstruction from Autonomous Quadrocopters Computer Science & Mathematics TU Munich Martin Oswald, Jakob Engel, Christian Kerl, Frank Steinbrücker, Jan Stühmer & Jürgen Sturm Autonomous Quadrocopters
Dense Tracking and Mapping for Autonomous Quadrocopters. Jürgen Sturm
Computer Vision Group Prof. Daniel Cremers Dense Tracking and Mapping for Autonomous Quadrocopters Jürgen Sturm Joint work with Frank Steinbrücker, Jakob Engel, Christian Kerl, Erik Bylow, and Daniel Cremers
Semi-Dense Direct SLAM
Computer Vision Group Technical University of Munich Jakob Engel Jakob Engel, Daniel Cremers David Caruso, Thomas Schöps, Lukas von Stumberg, Vladyslav Usenko, Jörg Stückler, Jürgen Sturm Technical University
Scanning and Printing Objects in 3D
Scanning and Printing Objects in 3D Dr. Jürgen Sturm metaio GmbH (formerly Technical University of Munich) My Research Areas Visual navigation for mobile robots RoboCup Kinematic Learning Articulated Objects
Deep Incremental Scene Understanding. Federico Tombari & Christian Rupprecht Technical University of Munich, Germany
Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical University of Munich, Germany C. Couprie et al. "Toward Real-time Indoor Semantic Segmentation Using Depth Information"
Direct Methods in Visual Odometry
Direct Methods in Visual Odometry July 24, 2017 Direct Methods in Visual Odometry July 24, 2017 1 / 47 Motivation for using Visual Odometry Wheel odometry is affected by wheel slip More accurate compared
Project Updates Short lecture Volumetric Modeling +2 papers
Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23
Jakob Engel, Thomas Schöps, Daniel Cremers Technical University Munich. LSD-SLAM: Large-Scale Direct Monocular SLAM
Computer Vision Group Technical University of Munich Jakob Engel LSD-SLAM: Large-Scale Direct Monocular SLAM Jakob Engel, Thomas Schöps, Daniel Cremers Technical University Munich Monocular Video Engel,
Colored Point Cloud Registration Revisited Supplementary Material
Colored Point Cloud Registration Revisited Supplementary Material Jaesik Park Qian-Yi Zhou Vladlen Koltun Intel Labs A. RGB-D Image Alignment Section introduced a joint photometric and geometric objective
RGBD Point Cloud Alignment Using Lucas-Kanade Data Association and Automatic Error Metric Selection
IEEE TRANSACTIONS ON ROBOTICS, VOL. 31, NO. 6, DECEMBER 15 1 RGBD Point Cloud Alignment Using Lucas-Kanade Data Association and Automatic Error Metric Selection Brian Peasley and Stan Birchfield, Senior
Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences
Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences Frank Steinbru cker, Christian Kerl, Ju rgen Sturm, and Daniel Cremers Technical University of Munich Boltzmannstrasse 3, 85748
Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences
2013 IEEE International Conference on Computer Vision Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences Frank Steinbrücker, Christian Kerl, Jürgen Sturm, and Daniel Cremers Technical
Robust Online 3D Reconstruction Combining a Depth Sensor and Sparse Feature Points
2016 23rd International Conference on Pattern Recognition (ICPR) Cancún Center, Cancún, México, December 4-8, 2016 Robust Online 3D Reconstruction Combining a Depth Sensor and Sparse Feature Points Erik
Geometric Reconstruction Dense reconstruction of scene geometry
Lecture 5. Dense Reconstruction and Tracking with Real-Time Applications Part 2: Geometric Reconstruction Dr Richard Newcombe and Dr Steven Lovegrove Slide content developed from: [Newcombe, Dense Visual
Mobile Point Fusion. Real-time 3d surface reconstruction out of depth images on a mobile platform
Mobile Point Fusion Real-time 3d surface reconstruction out of depth images on a mobile platform Aaron Wetzler Presenting: Daniel Ben-Hoda Supervisors: Prof. Ron Kimmel Gal Kamar Yaron Honen Supported
FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs
FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs W. Nicholas Greene Robust Robotics Group, MIT CSAIL LPM Workshop IROS 2017 September 28, 2017 with Nicholas Roy 1
A Real-Time RGB-D Registration and Mapping Approach by Heuristically Switching Between Photometric And Geometric Information
A Real-Time RGB-D Registration and Mapping Approach by Heuristically Switching Between Photometric And Geometric Information The 17th International Conference on Information Fusion (Fusion 2014) Khalid
Simultaneous Localization and Calibration: Self-Calibration of Consumer Depth Cameras
Simultaneous Localization and Calibration: Self-Calibration of Consumer Depth Cameras Qian-Yi Zhou Vladlen Koltun Abstract We describe an approach for simultaneous localization and calibration of a stream
CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras
CuFusion2: Accurate and Denoised Volumetric 3D Object Reconstruction Using Depth Cameras Chen Zhang College of Computer Science and Technology, Zhejiang University, Hangzhou 30027, China; Corresponding
OpenStreetSLAM: Global Vehicle Localization using OpenStreetMaps
OpenStreetSLAM: Global Vehicle Localization using OpenStreetMaps Georgios Floros, Benito van der Zander and Bastian Leibe RWTH Aachen University, Germany http://www.vision.rwth-aachen.de floros@vision.rwth-aachen.de
DNA-SLAM: Dense Noise Aware SLAM for ToF RGB-D Cameras
DNA-SLAM: Dense Noise Aware SLAM for ToF RGB-D Cameras Oliver Wasenmüller, Mohammad Dawud Ansari, Didier Stricker German Research Center for Artificial Intelligence (DFKI) University of Kaiserslautern
arxiv: v1 [cs.ro] 23 Nov 2015
![arxiv: v1 [cs.ro] 23 Nov 2015](/thumbs/73/68449431.jpg "arxiv: v1 [cs.ro] 23 Nov 2015")
Multi-Volume High Resolution RGB-D Mapping with Dynamic Volume Placement Michael Salvato 1, Ross Finman 1, and John J. Leonard 1 arxiv:1511.07106v1 [cs.ro] 23 Nov 2015 I. ABSTRACT Abstract We present a
Deep Models for 3D Reconstruction
Deep Models for 3D Reconstruction Andreas Geiger Autonomous Vision Group, MPI for Intelligent Systems, Tübingen Computer Vision and Geometry Group, ETH Zürich October 12, 2017 Max Planck Institute for
Computer Vision 2 Lecture 1
Computer Vision 2 Lecture 1 Introduction (14.04.2016) leibe@vision.rwth-aachen.de, stueckler@vision.rwth-aachen.de RWTH Aachen University, Computer Vision Group http://www.vision.rwth-aachen.de Organization
Real-Time Vision-Based State Estimation and (Dense) Mapping
Real-Time Vision-Based State Estimation and (Dense) Mapping Stefan Leutenegger IROS 2016 Workshop on State Estimation and Terrain Perception for All Terrain Mobile Robots The Perception-Action Cycle in
A Benchmark for RGB-D Visual Odometry, 3D Reconstruction and SLAM
2014 IEEE International Conference on Robotics & Automation (ICRA) Hong Kong Convention and Exhibition Center May 31 - June 7, 2014. Hong Kong, China A Benchmark for RGB-D Visual Odometry, 3D Reconstruction
3D Object Recognition and Scene Understanding from RGB-D Videos. Yu Xiang Postdoctoral Researcher University of Washington
3D Object Recognition and Scene Understanding from RGB-D Videos Yu Xiang Postdoctoral Researcher University of Washington 1 2 Act in the 3D World Sensing & Understanding Acting Intelligent System 3D World
Tracking an RGB-D Camera Using Points and Planes
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Tracking an RGB-D Camera Using Points and Planes Ataer-Cansizoglu, E.; Taguchi, Y.; Ramalingam, S.; Garaas, T. TR2013-106 December 2013 Abstract
Oblique Image Processing in SURE - First Experiments and Results
EuroSDR / ISPRS Workshop, Southampton 2015 Oblique Image Processing in SURE - First Experiments and Results Mathias Rothermel Outline» General Workflow» 2.5D Meshing» Mesh Texturing» 3D Workflow» First
3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller
3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
3D Object Representations. COS 526, Fall 2016 Princeton University
3D Object Representations COS 526, Fall 2016 Princeton University 3D Object Representations How do we... Represent 3D objects in a computer? Acquire computer representations of 3D objects? Manipulate computer
Photorealistic 3D Mapping of Indoors by RGB-D Scanning Process
Photorealistic 3D Mapping of Indoors by RGB-D Scanning Process Tommi Tykkälä 1, Andrew I. Comport 2, and Joni-Kristian Kämäräinen 3 Abstract In this work, a RGB-D input stream is utilized for GPU-boosted
Simultaneously Estimation of Super-Resolution Images and Depth Maps from Low Resolution Sensors
Simultaneously Estimation of Super-Resolution Images and Depth Maps from Low Resolution Sensors Daniel B. Mesquita, Erickson R. Nascimento, Mario F. M. Campos Computer Science Department Universidade Federal
Multi-Volume High Resolution RGB-D Mapping with Dynamic Volume Placement. Michael Salvato
Multi-Volume High Resolution RGB-D Mapping with Dynamic Volume Placement by Michael Salvato S.B. Massachusetts Institute of Technology (2013) Submitted to the Department of Electrical Engineering and Computer
ElasticFusion: Dense SLAM Without A Pose Graph
Robotics: Science and Systems 2015 Rome, Italy, July 13-17, 2015 ElasticFusion: Dense SLAM Without A Pose Graph Thomas Whelan*, Stefan Leutenegger*, Renato F. Salas-Moreno, Ben Glocker and Andrew J. Davison*
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting Robert Maier 1,2 Kihwan Kim 1 Daniel Cremers 2 Jan Kautz 1 Matthias Nießner 2,3
MonoRGBD-SLAM: Simultaneous Localization and Mapping Using Both Monocular and RGBD Cameras
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com MonoRGBD-SLAM: Simultaneous Localization and Mapping Using Both Monocular and RGBD Cameras Yousif, K.; Taguchi, Y.; Ramalingam, S. TR2017-068
CopyMe3D: Scanning and Printing Persons in 3D
CopyMe3D: Scanning and Printing Persons in 3D Sturm, Jürgen; Bylow, Erik; Kahl, Fredrik; Cremers, Daniel Published in: Lecture Notes in Computer Science Vol. 8142 (Pattern Recognition, 35th German Conference,
Introduction to Mobile Robotics Techniques for 3D Mapping
Introduction to Mobile Robotics Techniques for 3D Mapping Wolfram Burgard, Michael Ruhnke, Bastian Steder 1 Why 3D Representations Robots live in the 3D world. 2D maps have been applied successfully for
Approximate Surface Reconstruction and Registration for RGB-D SLAM
European Conference on Mobile Robotics (ECMR), Lincoln, UK, September 2015 Approximate Surface Reconstruction and Registration for RGB-D SLAM Dirk Holz and Sven Behnke Abstract RGB-D cameras have attracted
3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.
3D Reconstruction with Tango Ivan Dryanovski, Google Inc. Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools Problem formulation
Integrating Depth and Color Cues for Dense Multi-Resolution Scene Mapping Using RGB-D Cameras
In Proc. of the IEEE Int. Conf. on Multisensor Fusion and Information Integration (MFI), Hamburg, Germany, 2012 Integrating Depth and Color Cues for Dense Multi-Resolution Scene Mapping Using RGB-D Cameras
Hybrids Mixed Approaches
Hybrids Mixed Approaches Stephan Weiss Computer Vision Group NASA-JPL / CalTech Stephan.Weiss@ieee.org (c) 2013. Government sponsorship acknowledged. Outline Why mixing? Parallel Tracking and Mapping Benefits
CuFusion: Accurate real-time camera tracking and volumetric scene reconstruction with a cuboid
CuFusion: Accurate real-time camera tracking and volumetric scene reconstruction with a cuboid Chen Zhang * and Yu Hu College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China;
Reconstructing Reflective and Transparent Surfaces from Epipolar Plane Images
Sven Wanner and Bastian Goldlücke Heidelberg Collaboratory for Image Processing 1. Light Fields and Epipolar Plane Images 1. Light Fields and Epipolar Plane Images Light Fields as a dense sampling of a
Optimized KinectFusion Algorithm for 3D Scanning Applications
Optimized KinectFusion Algorithm for 3D Scanning Applications Faraj Alhwarin, Stefan Schiffer, Alexander Ferrein and Ingrid Scholl Mobile Autonomous Systems & Cognitive Robotics Institute (MASCOR), FH
Perceiving the 3D World from Images and Videos. Yu Xiang Postdoctoral Researcher University of Washington
Perceiving the 3D World from Images and Videos Yu Xiang Postdoctoral Researcher University of Washington 1 2 Act in the 3D World Sensing & Understanding Acting Intelligent System 3D World 3 Understand
arxiv: v1 [cs.cv] 1 Aug 2017
![arxiv: v1 [cs.cv] 1 Aug 2017](/thumbs/94/121686586.jpg "arxiv: v1 [cs.cv] 1 Aug 2017")
Dense Piecewise Planar RGB-D SLAM for Indoor Environments Phi-Hung Le and Jana Kosecka arxiv:1708.00514v1 [cs.cv] 1 Aug 2017 Abstract The paper exploits weak Manhattan constraints to parse the structure
Reconstructing Articulated Rigged Models from RGB-D Videos
Reconstructing Articulated Rigged Models from RGB-D Videos Dimitrios Tzionas 1,2 Juergen Gall 1 1 University of Bonn 2 MPI for Intelligent Systems {tzionas,gall}@iai.uni-bonn.de Abstract. Although commercial
Outline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping
Outline CSE 576 KinectFusion: Real-Time Dense Surface Mapping and Tracking PhD. work from Imperial College, London Microsoft Research, Cambridge May 6, 2013 1 Why we re interested in Real-Time tracking
3D Colored Model Generation Based on Multiview Textures and Triangular Mesh
3D Colored Model Generation Based on Multiview Textures and Triangular Mesh Lingni Ma, Luat Do, Egor Bondarev and Peter H. N. de With Department of Electrical Engineering, Eindhoven University of Technology
Stereo and Kinect fusion for continuous. 3D reconstruction and visual odometry
1 Stereo and Kinect fusion for continuous 3D reconstruction and visual odometry Ozgur YILMAZ #*1, Fatih KARAKUS *2 # Department of Computer Engineering, Turgut Özal University Ankara, TURKEY 1 ozyilmaz@turgutozal.edu.tr
Combining Depth Fusion and Photometric Stereo for Fine-Detailed 3D Models
Combining Depth Fusion and Photometric Stereo for Fine-Detailed 3D Models Erik Bylow 1[0000 0002 6665 1637], Robert Maier 2[0000 0003 4428 1089], Fredrik Kahl 3[0000 0001 9835 3020], and Carl Olsson 1,3[0000
Joint Multiview Segmentation and Localization of RGB-D Images using Depth-Induced Silhouette Consistency
Joint Multiview Segmentation and Localization of RGB-D Images using Depth-Induced Silhouette Consistency Chi Zhang 1,3 Zhiwei Li 2 Rui Cai 2 Hongyang Chao 1,3 Yong Rui 2 1 Sun Yat-Sen University 2 Microsoft
arxiv: v2 [cs.cv] 26 Aug 2018
![arxiv: v2 [cs.cv] 26 Aug 2018](/thumbs/91/105044371.jpg "arxiv: v2 [cs.cv] 26 Aug 2018")
A Synchronized Stereo and Plenoptic Visual Odometry Dataset Niclas Zeller 1,2, Franz Quint 2, and Uwe Stilla 1 1 Technische Universität München niclas.zeller@tum.de, stilla@tum.de 2 Karlsruhe University
LEARNING RIGIDITY IN DYNAMIC SCENES FOR SCENE FLOW ESTIMATION
LEARNING RIGIDITY IN DYNAMIC SCENES FOR SCENE FLOW ESTIMATION Kihwan Kim, Senior Research Scientist Zhaoyang Lv, Kihwan Kim, Alejandro Troccoli, Deqing Sun, James M. Rehg, Jan Kautz CORRESPENDECES IN COMPUTER
Signed Distance Function Representation, Tracking, and Mapping. Tanner Schmidt
Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations SDF fusion SDF tracking Related research - KinectFusion Patch Volumes
Robustness to Lighting Variations: An RGB-D Indoor Visual Odometry Using Line Segments
Robustness to Lighting Variations: An RGB-D Indoor Visual Odometry Using Line Segments Yan Lu and Dezhen Song Abstract Large lighting variation challenges all visual odometry methods, even with RGB-D cameras.
ROBUST LOOP CLOSURES FOR SCENE RECONSTRUCTION BY COMBINING ODOMETRY AND VISUAL CORRESPONDENCES
ROBUST LOOP CLOSURES FOR SCENE RECONSTRUCTION BY COMBINING ODOMETRY AND VISUAL CORRESPONDENCES Zakaria Laskar, Sami Huttunen, Daniel Herrera C., Esa Rahtu and Juho Kannala University of Oulu, Finland Aalto
Deformation-based Loop Closure for Large Scale Dense RGB-D SLAM
2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) November 3-7, 2013. Tokyo, Japan Deformation-based Loop Closure for Large Scale Dense RGB-D SLAM Thomas Whelan 1, Michael
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,
Volumetric 3D Mapping in Real-Time on a CPU
Volumetric 3D Mapping in Real-Time on a CPU Frank Steinbrücker, Jürgen Sturm, and Daniel Cremers Abstract In this paper we propose a novel volumetric multi-resolution mapping system for RGB-D images that
KinectFusion: Real-Time Dense Surface Mapping and Tracking
KinectFusion: Real-Time Dense Surface Mapping and Tracking Gabriele Bleser Thanks to Richard Newcombe for providing the ISMAR slides Overview General: scientific papers (structure, category) KinectFusion:
Kintinuous: Spatially Extended KinectFusion Thomas Whelan, Michael Kaess, Maurice Fallon, Hordur Johannsson, John Leonard, and John McDonald
Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-2012-020 July 19, 2012 Kintinuous: Spatially Extended KinectFusion Thomas Whelan, Michael Kaess, Maurice Fallon, Hordur
Multiframe Scene Flow with Piecewise Rigid Motion. Vladislav Golyanik,, Kihwan Kim, Robert Maier, Mathias Nießner, Didier Stricker and Jan Kautz
Multiframe Scene Flow with Piecewise Rigid Motion Vladislav Golyanik,, Kihwan Kim, Robert Maier, Mathias Nießner, Didier Stricker and Jan Kautz Scene Flow. 2 Scene Flow. 3 Scene Flow. Scene Flow Estimation:
Feature-based RGB-D camera pose optimization for real-time 3D reconstruction
Computational Visual Media DOI 10.1007/s41095-016-0072-2 Research Article Feature-based RGB-D camera pose optimization for real-time 3D reconstruction Chao Wang 1, Xiaohu Guo 1 ( ) c The Author(s) 2016.
Robust Real-Time Visual Odometry for Dense RGB-D Mapping
Robust Real-Time Visual Odometry for Dense RGB-D Mapping Thomas Whelan 1, Hordur Johannsson, Michael Kaess, John J. Leonard and John McDonald 1 Abstract This paper describes extensions to the Kintinuous
o Basic signal processing o Filtering, resampling, warping,... Rendering o Polygon rendering pipeline o Ray tracing Modeling
Background COS526: Advanced Computer Graphics Tom Funkhouser Fall 2010 Image Processing o Basic signal processing o Filtering, resampling, warping,... Rendering o Polygon rendering pipeline o Ray tracing
Vol agile avec des micro-robots volants contrôlés par vision
Vol agile avec des micro-robots volants contrôlés par vision From Active Perception to Event-based Vision Henri Rebecq from Prof. Davide Scaramuzza s lab GT UAV 17 Novembre 2016, Paris Davide Scaramuzza
Lecture 10 Dense 3D Reconstruction
Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,
3D Line Segments Extraction from Semi-dense SLAM
3D Line Segments Extraction from Semi-dense SLAM Shida He Xuebin Qin Zichen Zhang Martin Jagersand University of Alberta Abstract Despite the development of Simultaneous Localization and Mapping (SLAM),
3D Imaging from Video and Planar Radiography
3D Imaging from Video and Planar Radiography Julien Pansiot and Edmond Boyer Morpheo, Inria Grenoble Rhône-Alpes, France International Conference on Medical Image Computing and Computer Assisted Intervention
Patch-Based Optimization for Image-Based Texture Mapping
Patch-Based Optimization for Image-Based Texture Mapping SAI BI, University of California, San Diego NIMA KHADEMI KALANTARI, University of California, San Diego RAVI RAMAMOORTHI, University of California,
Photorealistic Texturing of Human Busts Reconstructions
Abstract Photorealistic Texturing of Human Busts Reconstructions Christoph HEINDL*, Sharath Chandra AKKALADEVI, Harald BAUER PROFACTOR GmbH, Steyr, Austria DOI: 10.15221/16.225 http://dx.doi.org/10.15221/16.225
Local-Level 3D Deep Learning. Andy Zeng
Local-Level 3D Deep Learning Andy Zeng 1 Matching 3D Data Image Credits: Song and Xiao, Tevs et al. 2 Matching 3D Data Reconstruction Image Credits: Song and Xiao, Tevs et al. 3 Matching 3D Data Reconstruction
Visual-Inertial Localization and Mapping for Robot Navigation
Visual-Inertial Localization and Mapping for Robot Navigation Dr. Guillermo Gallego Robotics & Perception Group University of Zurich Davide Scaramuzza University of Zurich - http://rpg.ifi.uzh.ch Mocular,
Autonomous Navigation for Flying Robots
Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 7.2: Visual Odometry Jürgen Sturm Technische Universität München Cascaded Control Robot Trajectory 0.1 Hz Visual
3DLite: Towards Commodity 3D Scanning for Content Creation
3DLite: Towards Commodity 3D Scanning for Content Creation JINGWEI HUANG, Stanford University ANGELA DAI, Stanford University and Technical University of Munich LEONIDAS GUIBAS, Stanford University MATTHIAS
PL-SVO: Semi-Direct Monocular Visual Odometry by Combining Points and Line Segments
2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Daejeon Convention Center October 9-14, 2016, Daejeon, Korea PL-SVO: Semi-Direct Monocular Visual Odometry by Combining Points
WHITE PAPER Scale-Awareness of Light Field Camera Based Visual Odometry
WHITE PAPER www.visteon.com Scale-Awareness of Light Field Camera Based isual Odometry Scale-Awareness of Light Field Camera based isual Odometry Niclas Zeller 1,2,3 and Franz Quint 2 and Uwe Stilla 1
Nonrigid Surface Modelling. and Fast Recovery. Department of Computer Science and Engineering. Committee: Prof. Leo J. Jia and Prof. K. H.
Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering May 11, 2007 1 2
Virtualized Reality Using Depth Camera Point Clouds
Virtualized Reality Using Depth Camera Point Clouds Jordan Cazamias Stanford University jaycaz@stanford.edu Abhilash Sunder Raj Stanford University abhisr@stanford.edu Abstract We explored various ways
Visual Perception for Robots
Visual Perception for Robots Sven Behnke Computer Science Institute VI Autonomous Intelligent Systems Our Cognitive Robots Complete systems for example scenarios Equipped with rich sensors Flying robot
Application questions. Theoretical questions
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list
Live Metric 3D Reconstruction on Mobile Phones ICCV 2013
Live Metric 3D Reconstruction on Mobile Phones ICCV 2013 Main Contents 1. Target & Related Work 2. Main Features of This System 3. System Overview & Workflow 4. Detail of This System 5. Experiments 6.
arxiv: v1 [cs.ro] 24 Nov 2018
![arxiv: v1 [cs.ro] 24 Nov 2018](/thumbs/91/105453267.jpg "arxiv: v1 [cs.ro] 24 Nov 2018")
BENCHMARKING AND COMPARING POPULAR VISUAL SLAM ALGORITHMS arxiv:1811.09895v1 [cs.ro] 24 Nov 2018 Amey Kasar Department of Electronics and Telecommunication Engineering Pune Institute of Computer Technology
arxiv: v2 [cs.cv] 9 Sep 2016
![arxiv: v2 [cs.cv] 9 Sep 2016](/thumbs/93/112016570.jpg "arxiv: v2 [cs.cv] 9 Sep 2016")
arxiv:1609.01371v2 [cs.cv] 9 Sep 2016 Reconstructing Articulated Rigged Models from RGB-D Videos Dimitrios Tzionas 1,2 Juergen Gall 1 1 University of Bonn 2 MPI for Intelligent Systems {tzionas,gall}@iai.uni-bonn.de
Semi-Direct EKF-based Monocular Visual-Inertial Odometry
Semi-Direct EKF-based Monocular Visual-Inertial Odometry Petri Tanskanen 1, Tobias Naegeli 2, Marc Pollefeys 1 and Otmar Hilliges 2 Abstract We propose a novel monocular visual inertial odometry algorithm
Visual Navigation for Flying Robots Exploration, Multi-Robot Coordination and Coverage
Computer Vision Group Prof. Daniel Cremers Visual Navigation for Flying Robots Exploration, Multi-Robot Coordination and Coverage Dr. Jürgen Sturm Agenda for Today Exploration with a single robot Coordinated
Application of LSD-SLAM for Visualization Temperature in Wide-area Environment
Application of LSD-SLAM for Visualization Temperature in Wide-area Environment Masahiro Yamaguchi 1, Hideo Saito 1 and Shoji Yachida 2 1 Graduate School of Science and Technology, Keio University, 3-14-1
L10 Layered Depth Normal Images. Introduction Related Work Structured Point Representation Boolean Operations Conclusion
L10 Layered Depth Normal Images Introduction Related Work Structured Point Representation Boolean Operations Conclusion 1 Introduction Purpose: using the computational power on GPU to speed up solid modeling
RGB-D Object Modelling for Object Recognition and Tracking
Margin requirements for first page 72 pt 1 in 25.4 mm RGB-D Object Modelling for Object Recognition and Tracking Johann Prankl, Aitor Aldoma, Alexander Svejda and Markus Vincze Abstract This work presents
3D Scene Reconstruction with a Mobile Camera
3D Scene Reconstruction with a Mobile Camera 1 Introduction Robert Carrera and Rohan Khanna Stanford University: CS 231A Autonomous supernumerary arms, or "third arms", while still unconventional, hold
Ping Tan. Simon Fraser University
Ping Tan Simon Fraser University Photos vs. Videos (live photos) A good photo tells a story Stories are better told in videos Videos in the Mobile Era (mobile & share) More videos are captured by mobile
Fast Odometry and Scene Flow from RGB-D Cameras based on Geometric Clustering
Fast Odometry and Scene Flow from RGB-D Cameras based on Geometric Clustering Mariano Jaimez 1,2, Christian Kerl 2, Javier Gonzalez-Jimenez 1 and Daniel Cremers 2 Abstract In this paper we propose an efficient
RGB-D Edge Detection and Edge-based Registration
23 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) November 3-7, 23. Tokyo, Japan RGB-D Edge Detection and Edge-based Registration Changhyun Choi, Alexander J. B. Trevor, and