Deep Models for 3D Reconstruction
|
|
- Mavis Patrick
- 6 years ago
- Views:
Transcription
1 Deep Models for 3D Reconstruction Andreas Geiger Autonomous Vision Group, MPI for Intelligent Systems, Tübingen Computer Vision and Geometry Group, ETH Zürich October 12, 2017 Max Planck Institute for Intelligent Systems Autonomous Vision Group
2 3D Reconstruction [Furukawa & Hernandez: Multi-View Stereo: A Tutorial] Task: Given a set of 2D images Reconstruct 3D shape of object/scene 2
3 3D Reconstruction Pipeline Input Images 3
4 3D Reconstruction Pipeline Input Images Camera Poses 3
5 3D Reconstruction Pipeline Input Images Camera Poses Dense Correspondences 3
6 3D Reconstruction Pipeline Input Images Camera Poses Dense Correspondences Depth Maps 3
7 3D Reconstruction Pipeline Input Images Camera Poses Dense Correspondences Depth Map Fusion Depth Maps 3
8 3D Reconstruction Pipeline Input Images Camera Poses Dense Correspondences 3D Reconstruction Depth Map Fusion Depth Maps 3
9 3D Reconstruction Pipeline Input Images Camera Poses Dense Correspondences 3D Reconstruction Depth Map Fusion Depth Maps 3
10 Large 3D Datasets and Repositories [Newcombe et al., 2011] [Choi et al., 2011] [Dai et al., 2017] [Wu et al., 2015] [Chang et al., 2015] [Chang et al., 2017] 4
11 Can we learn 3D Reconstruction from Data?
12 OctNet: Learning Deep 3D Representations at High Resolutions [Riegler, Ulusoy, & Geiger, CVPR 2017]
13 Deep Learning in 2D [LeCun, 1998] 7
14 Deep Learning in 3D 8
15 Deep Learning in 3D Existing 3D networks limited to 32 3 voxels 8
16 3D Data is often Sparse [Geiger et al., 2012] 9
17 3D Data is often Sparse [Li et al., 2016] 9
18 3D Data is often Sparse [Li et al., 2016] Can we exploit sparsity for efficient deep learning? 9
19 Network Activations Layer 1: 32 3 Layer 2: 16 3 Layer 3:
20 Network Activations Layer 1: 32 3 Layer 2: 16 3 Layer 3:
21 Network Activations Layer 1: 32 3 Layer 2: 16 3 Layer 3: 8 3 Idea: Partition space adaptively based on sparse input 10
22 Convolution 11
23 Convolution
24 Convolution
25 Convolution
26 Convolution
27 Convolution
28 Convolution
29 Convolution
30 Convolution
31 Convolution
32 Convolution
33 Convolution
34 Convolution
35 Convolution
36 Convolution
37 Convolution
38 Convolution 11
39 Convolution 11
40 Convolution Differentiable allows for end-to-end learning 11
41 Efficient Convolution This operation can be implemented very efficiently: 4 different cases First case requires only 1 evaluation! 12
42 Pooling 13
43 Pooling 13
44 Pooling 13
45 Pooling Unpooling operation defined similarly 13
46 Results: 3D Shape Classification Airplane Convolution and Pooling Convolution and Pooling Fully Conn. Fully Conn. 14
47 Results: 3D Shape Classification Memory [GB] OctNet DenseNet Input Resolution 15
48 Results: 3D Shape Classification Runtime [s] OctNet DenseNet Input Resolution 15
49 Results: 3D Shape Classification Accuracy OctNet 0.70 DenseNet Input Resolution Input: voxelized meshes from ModelNet 16
50 Results: 3D Shape Classification Accuracy OctNet 1 OctNet 2 OctNet Input Resolution Input: voxelized meshes from ModelNet 16
51 Results: 3D Shape Classification 17
52 Results: 3D Semantic Labeling Input I Prediction Dataset: RueMonge
53 Results: 3D Semantic Labeling Skip Skip Convolution and Pooling Convolution and Pooling Unpooling and Conv. Unpooling and Conv. Decoder octree structure copied from encoder 19
54 Results: 3D Semantic Labeling IoU [Riemenschneider et al., 2014] 42.3 [Martinovic et al., 2015] 52.2 [Gadde et al., 2016] 54.4 OctNet OctNet OctNet
55 OctNetFusion: Learning Depth Fusion from Data [Riegler, Ulusoy, Bischof & Geiger, 3DV 2017]
56 Volumetric Fusion d i+1 (p) = w i(p)d i (p) + ŵ(p) ˆd(p) w i (p) + ŵ(p) w i+1 (p) = w i (p) + ŵ(p) p R 3 : voxel location d: distance, w: weight [Curless and Levoy, SIGGRAPH 1996] 22
57 Volumetric Fusion Pros: Simple, fast, easy to implement Defacto gold standard (KinectFusion, Voxel Hashing,...) Ground Truth Volumetric Fusion 23
58 Volumetric Fusion Pros: Simple, fast, easy to implement Defacto gold standard (KinectFusion, Voxel Hashing,...) Cons: Requires many redundant views to reduce noise Can t handle outliers / complete missing surfaces Ground Truth Volumetric Fusion 23
59 TV-L1 Fusion Pros: Prior on surface area Noise reduction Ground Truth Volumetric Fusion TV-L1 Fusion 23
60 TV-L1 Fusion Pros: Prior on surface area Noise reduction Cons: Simplistic local prior (penalizes surface area, shrinking bias) Can t complete missing surfaces Ground Truth Volumetric Fusion TV-L1 Fusion 23
61 Learned Fusion Pros: Learn noise suppression from data Learn surface completion from data Ground Truth Volumetric Fusion TV-L1 Fusion OctNetFusion 23
62 Learned Fusion Pros: Learn noise suppression from data Learn surface completion from data Cons: Requires large 3D datasets for training How to scale to high resolutions? Ground Truth Volumetric Fusion TV-L1 Fusion OctNetFusion 23
63 Learning 3D Fusion Skip Skip Convolution and Pooling Convolution and Pooling Unpooling and Conv. Unpooling and Conv. Input Representation: TSDF Higher-order statistics Output Representation: Occupancy TSDF 24
64 Learning 3D Fusion Skip Skip Convolution and Pooling Convolution and Pooling Unpooling and Conv. Unpooling and Conv. What is the problem? 24
65 Learning 3D Fusion Skip Skip Convolution and Pooling Convolution and Pooling Unpooling and Conv. Unpooling and Conv. What is the problem? Octree structure unknown needs to be inferred as well! 24
66 OctNetFusion Architecture 64³ 64³ 64 Input Output Octree Structure 128³ Features 128³ 128 Input Output Octree Structure Features 256³ 256³ 256 Input Output 25
67 Results: Surface Reconstruction VolFus TV-L1 Ours Ground Truth
68 Results: Volumetric Completion [Firman, 2016] Ours Ground Truth 27
69 Thank you!
Geometric and Semantic 3D Reconstruction: Part 4A: Volumetric Semantic 3D Reconstruction. CVPR 2017 Tutorial Christian Häne UC Berkeley
Geometric and Semantic 3D Reconstruction: Part 4A: Volumetric Semantic 3D Reconstruction CVPR 2017 Tutorial Christian Häne UC Berkeley Dense Multi-View Reconstruction Goal: 3D Model from Images (Depth
More informationarxiv: v3 [cs.cv] 31 Oct 2017
OctNetFusion: Learning Depth Fusion from Data Gernot Riegler 1 Ali Osman Ulusoy 2 Horst Bischof 1 Andreas Geiger 2,3 1 Institute for Computer Graphics and Vision, Graz University of Technology 2 Autonomous
More informationLearning from 3D Data
Learning from 3D Data Thomas Funkhouser Princeton University* * On sabbatical at Stanford and Google Disclaimer: I am talking about the work of these people Shuran Song Andy Zeng Fisher Yu Yinda Zhang
More informationIntrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
More informationGeometric Reconstruction Dense reconstruction of scene geometry
Lecture 5. Dense Reconstruction and Tracking with Real-Time Applications Part 2: Geometric Reconstruction Dr Richard Newcombe and Dr Steven Lovegrove Slide content developed from: [Newcombe, Dense Visual
More informationCS468: 3D Deep Learning on Point Cloud Data. class label part label. Hao Su. image. May 10, 2017
CS468: 3D Deep Learning on Point Cloud Data class label part label Hao Su image. May 10, 2017 Agenda Point cloud generation Point cloud analysis CVPR 17, Point Set Generation Pipeline render CVPR 17, Point
More informationLearning to generate 3D shapes
Learning to generate 3D shapes Subhransu Maji College of Information and Computer Sciences University of Massachusetts, Amherst http://people.cs.umass.edu/smaji August 10, 2018 @ Caltech Creating 3D shapes
More informationPerceiving the 3D World from Images and Videos. Yu Xiang Postdoctoral Researcher University of Washington
Perceiving the 3D World from Images and Videos Yu Xiang Postdoctoral Researcher University of Washington 1 2 Act in the 3D World Sensing & Understanding Acting Intelligent System 3D World 3 Understand
More informationVolumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material
Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Charles R. Qi Hao Su Matthias Nießner Angela Dai Mengyuan Yan Leonidas J. Guibas Stanford University 1. Details
More informationProject Updates Short lecture Volumetric Modeling +2 papers
Volumetric Modeling Schedule (tentative) Feb 20 Feb 27 Mar 5 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Mar 12 Mar 19 Mar 26 Apr 2 Apr 9 Apr 16 Apr 23
More information3D Object Recognition and Scene Understanding from RGB-D Videos. Yu Xiang Postdoctoral Researcher University of Washington
3D Object Recognition and Scene Understanding from RGB-D Videos Yu Xiang Postdoctoral Researcher University of Washington 1 2 Act in the 3D World Sensing & Understanding Acting Intelligent System 3D World
More informationDeep Incremental Scene Understanding. Federico Tombari & Christian Rupprecht Technical University of Munich, Germany
Deep Incremental Scene Understanding Federico Tombari & Christian Rupprecht Technical University of Munich, Germany C. Couprie et al. "Toward Real-time Indoor Semantic Segmentation Using Depth Information"
More informationImage Based Reconstruction II
Image Based Reconstruction II Qixing Huang Feb. 2 th 2017 Slide Credit: Yasutaka Furukawa Image-Based Geometry Reconstruction Pipeline Last Lecture: Multi-View SFM Multi-View SFM This Lecture: Multi-View
More informationSemantic 3D Reconstruction of Heads Supplementary Material
Semantic 3D Reconstruction of Heads Supplementary Material Fabio Maninchedda1, Christian Ha ne2,?, Bastien Jacquet3,?, Amae l Delaunoy?, Marc Pollefeys1,4 1 ETH Zurich 2 UC Berkeley 3 Kitware SAS 4 Microsoft
More informationOutline. 1 Why we re interested in Real-Time tracking and mapping. 3 Kinect Fusion System Overview. 4 Real-time Surface Mapping
Outline CSE 576 KinectFusion: Real-Time Dense Surface Mapping and Tracking PhD. work from Imperial College, London Microsoft Research, Cambridge May 6, 2013 1 Why we re interested in Real-Time tracking
More informationDense 3D Reconstruction from Autonomous Quadrocopters
Dense 3D Reconstruction from Autonomous Quadrocopters Computer Science & Mathematics TU Munich Martin Oswald, Jakob Engel, Christian Kerl, Frank Steinbrücker, Jan Stühmer & Jürgen Sturm Autonomous Quadrocopters
More informationLecture 10 Dense 3D Reconstruction
Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,
More informationOctree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material Peak memory usage, GB 10 1 0.1 0.01 OGN Quadratic Dense Cubic Iteration time, s 10
More informationComputer Vision: Making machines see
Computer Vision: Making machines see Roberto Cipolla Department of Engineering http://www.eng.cam.ac.uk/~cipolla/people.html http://www.toshiba.eu/eu/cambridge-research- Laboratory/ Vision: what is where
More informationMulti-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
More informationMulti-View 3D Object Detection Network for Autonomous Driving
Multi-View 3D Object Detection Network for Autonomous Driving Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, Tian Xia CVPR 2017 (Spotlight) Presented By: Jason Ku Overview Motivation Dataset Network Architecture
More informationCVPR 2014 Visual SLAM Tutorial Kintinuous
CVPR 2014 Visual SLAM Tutorial Kintinuous kaess@cmu.edu The Robotics Institute Carnegie Mellon University Recap: KinectFusion [Newcombe et al., ISMAR 2011] RGB-D camera GPU 3D/color model RGB TSDF (volumetric
More informationDepth from Stereo. Dominic Cheng February 7, 2018
Depth from Stereo Dominic Cheng February 7, 2018 Agenda 1. Introduction to stereo 2. Efficient Deep Learning for Stereo Matching (W. Luo, A. Schwing, and R. Urtasun. In CVPR 2016.) 3. Cascade Residual
More information08 An Introduction to Dense Continuous Robotic Mapping
NAVARCH/EECS 568, ROB 530 - Winter 2018 08 An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy
More information3D Object Classification via Spherical Projections
3D Object Classification via Spherical Projections Zhangjie Cao 1,QixingHuang 2,andRamaniKarthik 3 1 School of Software Tsinghua University, China 2 Department of Computer Science University of Texas at
More informationDeconvolution Networks
Deconvolution Networks Johan Brynolfsson Mathematical Statistics Centre for Mathematical Sciences Lund University December 6th 2016 1 / 27 Deconvolution Neural Networks 2 / 27 Image Deconvolution True
More informationPredicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture David Eigen, Rob Fergus
Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture David Eigen, Rob Fergus Presented by: Rex Ying and Charles Qi Input: A Single RGB Image Estimate
More information3D Shape Segmentation with Projective Convolutional Networks
3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis 1 Melinos Averkiou 2 Subhransu Maji 1 Siddhartha Chaudhuri 3 1 University of Massachusetts Amherst 2 University of Cyprus
More informationObject Localization, Segmentation, Classification, and Pose Estimation in 3D Images using Deep Learning
Allan Zelener Dissertation Proposal December 12 th 2016 Object Localization, Segmentation, Classification, and Pose Estimation in 3D Images using Deep Learning Overview 1. Introduction to 3D Object Identification
More informationMatryoshka Networks: Predicting 3D Geometry via Nested Shape Layers
Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers Stephan R. Richter 1,2 Stefan Roth 1 1 TU Darmstadt 2 Intel Labs Figure 1. Exemplary shape reconstructions from a single by our Matryoshka
More information3D Shape Analysis with Multi-view Convolutional Networks. Evangelos Kalogerakis
3D Shape Analysis with Multi-view Convolutional Networks Evangelos Kalogerakis 3D model repositories [3D Warehouse - video] 3D geometry acquisition [KinectFusion - video] 3D shapes come in various flavors
More informationLecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza
Lecture 10 Multi-view Stereo (3D Dense Reconstruction) Davide Scaramuzza REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time, ICRA 14, by Pizzoli, Forster, Scaramuzza [M. Pizzoli, C. Forster,
More information3D model classification using convolutional neural network
3D model classification using convolutional neural network JunYoung Gwak Stanford jgwak@cs.stanford.edu Abstract Our goal is to classify 3D models directly using convolutional neural network. Most of existing
More informationSuper-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures
Super-Resolution Keyframe Fusion for 3D Modeling with High-Quality Textures Robert Maier, Jörg Stückler, Daniel Cremers International Conference on 3D Vision (3DV) October 2015, Lyon, France Motivation
More informationarxiv: v2 [cs.cv] 16 Nov 2016
OctNet: Learning Deep 3D Representations at High Resolutions Gernot Riegler 1 Ali Osman Ulusoy 2 Andreas Geiger 2,3 1 Institute for Computer Graphics and Vision, Graz University of Technology 2 Autonomous
More informationLecture 7: Semantic Segmentation
Semantic Segmentation CSED703R: Deep Learning for Visual Recognition (207F) Segmenting images based on its semantic notion Lecture 7: Semantic Segmentation Bohyung Han Computer Vision Lab. bhhanpostech.ac.kr
More informationPointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space Sikai Zhong February 14, 2018 COMPUTER SCIENCE Table of contents 1. PointNet 2. PointNet++ 3. Experiments 1 PointNet Property
More informationMULTI-LEVEL 3D CONVOLUTIONAL NEURAL NETWORK FOR OBJECT RECOGNITION SAMBIT GHADAI XIAN LEE ADITYA BALU SOUMIK SARKAR ADARSH KRISHNAMURTHY
MULTI-LEVEL 3D CONVOLUTIONAL NEURAL NETWORK FOR OBJECT RECOGNITION SAMBIT GHADAI XIAN LEE ADITYA BALU SOUMIK SARKAR ADARSH KRISHNAMURTHY Outline Object Recognition Multi-Level Volumetric Representations
More informationMesh from Depth Images Using GR 2 T
Mesh from Depth Images Using GR 2 T Mairead Grogan & Rozenn Dahyot School of Computer Science and Statistics Trinity College Dublin Dublin, Ireland mgrogan@tcd.ie, Rozenn.Dahyot@tcd.ie www.scss.tcd.ie/
More informationUnsupervised Learning of Spatiotemporally Coherent Metrics
Unsupervised Learning of Spatiotemporally Coherent Metrics Ross Goroshin, Joan Bruna, Jonathan Tompson, David Eigen, Yann LeCun arxiv 2015. Presented by Jackie Chu Contributions Insight between slow feature
More informationCAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision
CAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision Navaneet K L *, Priyanka Mandikal * and R. Venkatesh Babu Video Analytics Lab, IISc, Bangalore, India
More informationCAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision
CAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision Navaneet K L *, Priyanka Mandikal *, Mayank Agarwal, and R. Venkatesh Babu Video Analytics Lab, CDS, Indian
More informationDense Tracking and Mapping for Autonomous Quadrocopters. Jürgen Sturm
Computer Vision Group Prof. Daniel Cremers Dense Tracking and Mapping for Autonomous Quadrocopters Jürgen Sturm Joint work with Frank Steinbrücker, Jakob Engel, Christian Kerl, Erik Bylow, and Daniel Cremers
More informationDeep Marching Cubes: Learning Explicit Surface Representations
Deep Marching Cubes: Learning Explicit Surface Representations Yiyi Liao 1,2 Simon Donné 1,3 Andreas Geiger 1,4 1 Autonomous Vision Group, MPI for Intelligent Systems and University of Tübingen 2 Institute
More informationAn Empirical Study of Generative Adversarial Networks for Computer Vision Tasks
An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks Report for Undergraduate Project - CS396A Vinayak Tantia (Roll No: 14805) Guide: Prof Gaurav Sharma CSE, IIT Kanpur, India
More informationECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016
ECCV 2016 Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016 Fundamental Question What is a good vector representation of an object? Something that can be easily predicted from 2D
More informationUsing Faster-RCNN to Improve Shape Detection in LIDAR
Using Faster-RCNN to Improve Shape Detection in LIDAR TJ Melanson Stanford University Stanford, CA 94305 melanson@stanford.edu Abstract In this paper, I propose a method for extracting objects from unordered
More informationPOINT CLOUD DEEP LEARNING
POINT CLOUD DEEP LEARNING Innfarn Yoo, 3/29/28 / 57 Introduction AGENDA Previous Work Method Result Conclusion 2 / 57 INTRODUCTION 3 / 57 2D OBJECT CLASSIFICATION Deep Learning for 2D Object Classification
More informationProf. Trevor Darrell Lecture 18: Multiview and Photometric Stereo
C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 18: Multiview and Photometric Stereo Today Multiview stereo revisited Shape from large image collections Voxel Coloring Digital
More informationFast scene understanding and prediction for autonomous platforms. Bert De Brabandere, KU Leuven, October 2017
Fast scene understanding and prediction for autonomous platforms Bert De Brabandere, KU Leuven, October 2017 Who am I? MSc in Electrical Engineering at KU Leuven, Belgium Last year PhD student with Luc
More informationLearning visual odometry with a convolutional network
Learning visual odometry with a convolutional network Kishore Konda 1, Roland Memisevic 2 1 Goethe University Frankfurt 2 University of Montreal konda.kishorereddy@gmail.com, roland.memisevic@gmail.com
More informationLarge-Scale Point Cloud Classification Benchmark
Large-Scale Point Cloud Classification Benchmark www.semantic3d.net IGP & CVG, ETH Zürich www.semantic3d.net, info@semantic3d.net 7/6/2016 1 Timo Hackel Nikolay Savinov Ľubor Ladický Jan Dirk Wegner Konrad
More informationLive Metric 3D Reconstruction on Mobile Phones ICCV 2013
Live Metric 3D Reconstruction on Mobile Phones ICCV 2013 Main Contents 1. Target & Related Work 2. Main Features of This System 3. System Overview & Workflow 4. Detail of This System 5. Experiments 6.
More informationDeep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing Supplementary Material Introduction In this supplementary material, Section 2 details the 3D annotation for CAD models and real
More informationLocal Readjustment for High-Resolution 3D Reconstruction: Supplementary Material
Local Readjustment for High-Resolution 3D Reconstruction: Supplementary Material Siyu Zhu 1, Tian Fang 2, Jianxiong Xiao 3, and Long Quan 4 1,2,4 The Hong Kong University of Science and Technology 3 Princeton
More informationOctNet: Learning Deep 3D Representations at High Resolutions
OctNet: Learning Deep 3D Representations at High Resolutions Gernot Riegler 1 Ali Osman Ulusoy 2 Andreas Geiger 2,3 1 Institute for Computer Graphics and Vision, Graz University of Technology 2 Autonomous
More information3D Computer Vision. Depth Cameras. Prof. Didier Stricker. Oliver Wasenmüller
3D Computer Vision Depth Cameras Prof. Didier Stricker Oliver Wasenmüller Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More information3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.
3D Reconstruction with Tango Ivan Dryanovski, Google Inc. Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools Problem formulation
More informationRayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials Despoina Paschalidou 1,5 Ali Osman Ulusoy 2 Carolin Schmitt 1 Luc van Gool 3 Andreas Geiger 1,4 1 Autonomous Vision Group, MPI for Intelligent
More informationPoint2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network
Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network Xinhai Liu, Zhizhong Han,2, Yu-Shen Liu, Matthias Zwicker 2 School of Software,
More informationMulti-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
More informationSupplementary: Cross-modal Deep Variational Hand Pose Estimation
Supplementary: Cross-modal Deep Variational Hand Pose Estimation Adrian Spurr, Jie Song, Seonwook Park, Otmar Hilliges ETH Zurich {spurra,jsong,spark,otmarh}@inf.ethz.ch Encoder/Decoder Linear(512) Table
More informationDeep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
More information3D Object Representations. COS 526, Fall 2016 Princeton University
3D Object Representations COS 526, Fall 2016 Princeton University 3D Object Representations How do we... Represent 3D objects in a computer? Acquire computer representations of 3D objects? Manipulate computer
More informationDeep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing Supplementary Material
Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing Supplementary Material Chi Li, M. Zeeshan Zia 2, Quoc-Huy Tran 2, Xiang Yu 2, Gregory D. Hager, and Manmohan Chandraker 2 Johns
More informationarxiv: v1 [cs.cv] 28 Mar 2018
arxiv:1803.10409v1 [cs.cv] 28 Mar 2018 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation Angela Dai 1 Matthias Nießner 2 1 Stanford University 2 Technical University of Munich Fig.
More informationRTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017
RTSR: Enhancing Real-time H.264 Video Streaming using Deep Learning based Video Super Resolution Spring 2017 CS570 Project Presentation June 8, 2017 Team 16 Soomin Kim Leslie Tiong Youngki Kwon Insu Jang
More informationSupplementary Material for Sparsity Invariant CNNs
Supplementary Material for Sparsity Invariant CNNs Jonas Uhrig,1,2 Nick Schneider,1,3 Lukas Schneider 1,4 Uwe Franke 1 Thomas Brox 2 Andreas Geiger 4,5 1 Daimler R&D Sindelfingen 2 University of Freiburg
More informationCOMP 551 Applied Machine Learning Lecture 16: Deep Learning
COMP 551 Applied Machine Learning Lecture 16: Deep Learning Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551 Unless otherwise noted, all
More informationarxiv: v1 [cs.cv] 31 Mar 2018
arxiv:1804.00090v1 [cs.cv] 31 Mar 2018 FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans Chen Liu Jiaye Wu Washington University in St. Louis {chenliu,jiaye.wu}@wustl.edu Yasutaka
More informationLearning-based Localization
Learning-based Localization Eric Brachmann ECCV 2018 Tutorial on Visual Localization - Feature-based vs. Learned Approaches Torsten Sattler, Eric Brachmann Roadmap Machine Learning Basics [10min] Convolutional
More informationFloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans
FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans Chen Liu 1, Jiaye Wu 1, and Yasutaka Furukawa 2 1 Washington University in St. Louis, St. Louis, USA {chenliu,jiaye.wu}@wustl.edu
More informationarxiv: v2 [cs.cv] 27 Nov 2018
Learning 3D Shape Completion under Weak Supervision David Stutz Andreas Geiger arxiv:185.729v2 [cs.cv] 27 Nov 218 Abstract We address the problem of 3D shape completion from sparse and noisy point clouds,
More informationarxiv: v1 [cs.cv] 18 May 2018 Received: date / Accepted: date
Noname manuscript No. (will be inserted by the editor) Learning 3D Shape Completion under Weak Supervision David Stutz Andreas Geiger arxiv:185.729v1 [cs.cv] 18 May 218 Received: date / Accepted: date
More informationFrom Point Clouds to Mesh using Regression
From Point Clouds to Mesh using Regression L ubor Ladický ETH Zürich lubor.ladicky@inf.ethz.ch Fabio Maninchedda ETH Zürich fabiom@inf.ethz.ch Olivier Saurer ETH Zürich saurero@inf.ethz.ch Marc Pollefeys
More informationHierarchical Surface Prediction for 3D Object Reconstruction
Hierarchical Surface Prediction for 3D Object Reconstruction Christian Häne, Shubham Tulsiani, Jitendra Malik University of California, Berkeley {chaene, shubhtuls, malik}@eecs.berkeley.edu Abstract Recently,
More informationStereo Human Keypoint Estimation
Stereo Human Keypoint Estimation Kyle Brown Stanford University Stanford Intelligent Systems Laboratory kjbrown7@stanford.edu Abstract The goal of this project is to accurately estimate human keypoint
More informationScanning and Printing Objects in 3D Jürgen Sturm
Scanning and Printing Objects in 3D Jürgen Sturm Metaio (formerly Technical University of Munich) My Research Areas Visual navigation for mobile robots RoboCup Kinematic Learning Articulated Objects Quadrocopters
More informationGeometric Registration for Deformable Shapes 1.1 Introduction
Geometric Registration for Deformable Shapes 1.1 Introduction Overview Data Sources and Applications Problem Statement Overview Presenters Will Chang University of California at San Diego, USA Hao Li ETH
More informationA Patch Prior for Dense 3D Reconstruction in Man-Made Environments
A Patch Prior for Dense 3D Reconstruction in Man-Made Environments Christian Häne 1, Christopher Zach 2, Bernhard Zeisl 1, Marc Pollefeys 1 1 ETH Zürich 2 MSR Cambridge October 14, 2012 A Patch Prior for
More informationMulti-view 3D Models from Single Images with a Convolutional Network
Multi-view 3D Models from Single Images with a Convolutional Network Maxim Tatarchenko University of Freiburg Skoltech - 2nd Christmas Colloquium on Computer Vision Humans have prior knowledge about 3D
More informationVISION FOR AUTOMOTIVE DRIVING
VISION FOR AUTOMOTIVE DRIVING French Japanese Workshop on Deep Learning & AI, Paris, October 25th, 2017 Quoc Cuong PHAM, PhD Vision and Content Engineering Lab AI & MACHINE LEARNING FOR ADAS AND SELF-DRIVING
More informationDiscrete Optimization of Ray Potentials for Semantic 3D Reconstruction
Discrete Optimization of Ray Potentials for Semantic 3D Reconstruction Marc Pollefeys Joined work with Nikolay Savinov, Christian Haene, Lubor Ladicky 2 Comparison to Volumetric Fusion Higher-order ray
More information3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
3D Semantic Segmentation with Submanifold Sparse Convolutional Networks Benjamin Graham Facebook AI Research benjamingraham@fb.com Martin Engelcke University of Oxford martin@robots.ox.ac.uk Laurens van
More informationLearning to Reconstruct High-quality 3D Shapes with Cascaded Fully Convolutional Networks
Learning to Reconstruct High-quality 3D Shapes with Cascaded Fully Convolutional Networks * Yan-Pei Cao 1,2, * Zheng-Ning Liu 1, Zheng-Fei Kuang 1, Leif Kobbelt 3, Shi-Min Hu 1 * equal contribution 1 Tsinghua
More informationComputing the Stereo Matching Cost with CNN
University at Austin Figure. The of lefttexas column displays the left input image, while the right column displays the output of our stereo method. Examples are sorted by difficulty, with easy examples
More informationFLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs
FLaME: Fast Lightweight Mesh Estimation using Variational Smoothing on Delaunay Graphs W. Nicholas Greene Robust Robotics Group, MIT CSAIL LPM Workshop IROS 2017 September 28, 2017 with Nicholas Roy 1
More informationObject Detection. CS698N Final Project Presentation AKSHAT AGARWAL SIDDHARTH TANWAR
Object Detection CS698N Final Project Presentation AKSHAT AGARWAL SIDDHARTH TANWAR Problem Description Arguably the most important part of perception Long term goals for object recognition: Generalization
More informationFrom Orientation to Functional Modeling for Terrestrial and UAV Images
From Orientation to Functional Modeling for Terrestrial and UAV Images Helmut Mayer 1 Andreas Kuhn 1, Mario Michelini 1, William Nguatem 1, Martin Drauschke 2 and Heiko Hirschmüller 2 1 Visual Computing,
More informationCS5670: Computer Vision
CS5670: Computer Vision Noah Snavely Multi-view stereo Announcements Project 3 ( Autostitch ) due Monday 4/17 by 11:59pm Recommended Reading Szeliski Chapter 11.6 Multi-View Stereo: A Tutorial Furukawa
More informationarxiv: v2 [cs.cv] 28 Mar 2018
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans Angela Dai 1,3,5 Daniel Ritchie 2 Martin Bokeloh 3 Scott Reed 4 Jürgen Sturm 3 Matthias Nießner 5 1 Stanford University
More informationEncoder-Decoder Networks for Semantic Segmentation. Sachin Mehta
Encoder-Decoder Networks for Semantic Segmentation Sachin Mehta Outline > Overview of Semantic Segmentation > Encoder-Decoder Networks > Results What is Semantic Segmentation? Input: RGB Image Output:
More informationScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans Angela Dai 1,3,5 Daniel Ritchie 2 Martin Bokeloh 3 Scott Reed 4 Jürgen Sturm 3 Matthias Nießner 5 1 Stanford University
More informationPoint Cloud Completion of Foot Shape from a Single Depth Map for Fit Matching using Deep Learning View Synthesis
Point Cloud Completion of Foot Shape from a Single Depth Map for Fit Matching using Deep Learning View Synthesis Nolan Lunscher University of Waterloo 200 University Ave W. nlunscher@uwaterloo.ca John
More informationDeepIM: Deep Iterative Matching for 6D Pose Estimation - Supplementary Material
DeepIM: Deep Iterative Matching for 6D Pose Estimation - Supplementary Material Yi Li 1, Gu Wang 1, Xiangyang Ji 1, Yu Xiang 2, and Dieter Fox 2 1 Tsinghua University, BNRist 2 University of Washington
More informationFlow-Based Video Recognition
Flow-Based Video Recognition Jifeng Dai Visual Computing Group, Microsoft Research Asia Joint work with Xizhou Zhu*, Yuwen Xiong*, Yujie Wang*, Lu Yuan and Yichen Wei (* interns) Talk pipeline Introduction
More informationarxiv: v2 [cs.cv] 6 Nov 2017
Hierarchical Surface Prediction for 3D Object Reconstruction Christian Häne, Shubham Tulsiani, Jitendra Malik University of California, Berkeley {chaene, shubhtuls, malik}@eecs.berkeley.edu arxiv:1704.00710v2
More informationComputing 3D Geometry Directly From Range Images
Computing 3D Geometry Directly From Range Images Sarah F. Frisken and Ronald N. Perry Mitsubishi Electric Research Laboratories Geometry from Range Data A Classic Approach Subject Range images Range surfaces
More informationS7348: Deep Learning in Ford's Autonomous Vehicles. Bryan Goodman Argo AI 9 May 2017
S7348: Deep Learning in Ford's Autonomous Vehicles Bryan Goodman Argo AI 9 May 2017 1 Ford s 12 Year History in Autonomous Driving Today: examples from Stereo image processing Object detection Using RNN
More informationDeep Learning in Image Processing
Deep Learning in Image Processing Roland Memisevic University of Montreal & TwentyBN ICISP 2016 Roland Memisevic Deep Learning in Image Processing ICISP 2016 f 2? cathedral high-rise f 1 It s the features,
More information