Visual Computing TUM

Similar documents
Learning from 3D Data

3D Scene Understanding from RGB-D Images. Thomas Funkhouser

arxiv: v1 [cs.cv] 28 Mar 2018

Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material

Hide-and-Seek: Forcing a network to be Meticulous for Weakly-supervised Object and Action Localization

arxiv: v2 [cs.cv] 28 Mar 2018

CS6501: Deep Learning for Visual Recognition. Object Detection I: RCNN, Fast-RCNN, Faster-RCNN

Multi-view 3D Models from Single Images with a Convolutional Network

arxiv: v1 [cs.cv] 18 Sep 2017

ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

Deep Learning for Robust Normal Estimation in Unstructured Point Clouds. Alexandre Boulch. Renaud Marlet

Learning to generate 3D shapes

PIXELS TO VOXELS: MODELING VISUAL REPRESENTATION IN THE HUMAN BRAIN

Predicting ground-level scene Layout from Aerial imagery. Muhammad Hasan Maqbool

Object Detection in 3D Scenes Using CNNs in Multi-view Images

Object Localization, Segmentation, Classification, and Pose Estimation in 3D Images using Deep Learning

Photo-realistic Renderings for Machines Seong-heum Kim

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Charles R. Qi* Hao Su* Kaichun Mo Leonidas J. Guibas

ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes

arxiv: v1 [cs.cv] 27 Nov 2018

Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

CS468: 3D Deep Learning on Point Cloud Data. class label part label. Hao Su. image. May 10, 2017

JOINT DETECTION AND SEGMENTATION WITH DEEP HIERARCHICAL NETWORKS. Zhao Chen Machine Learning Intern, NVIDIA

Perceiving the 3D World from Images and Videos. Yu Xiang Postdoctoral Researcher University of Washington

AI Ukraine Point cloud labeling using machine learning

Cascade Region Regression for Robust Object Detection

LSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University

Constrained Convolutional Neural Networks for Weakly Supervised Segmentation. Deepak Pathak, Philipp Krähenbühl and Trevor Darrell

Seeing the unseen. Data-driven 3D Understanding from Single Images. Hao Su

COMP 551 Applied Machine Learning Lecture 16: Deep Learning

Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet.

arxiv: v1 [cs.cv] 10 Aug 2018

3D Object Recognition and Scene Understanding from RGB-D Videos. Yu Xiang Postdoctoral Researcher University of Washington

Using Faster-RCNN to Improve Shape Detection in LIDAR

Lecture 7: Semantic Segmentation

3D Deep Learning on Geometric Forms. Hao Su

Collision Experimental Data

SurfNet: Generating 3D shape surfaces using deep residual networks-supplementary Material

The Hilbert Problems of Computer Vision. Jitendra Malik UC Berkeley & Google, Inc.

What are we trying to achieve? Why are we doing this? What do we learn from past history? What will we talk about today?

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting

Scene Text Recognition for Augmented Reality. Sagar G V Adviser: Prof. Bharadwaj Amrutur Indian Institute Of Science

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image. Supplementary Material

3D Convolutional Neural Networks for Landing Zone Detection from LiDAR

ABC-CNN: Attention Based CNN for Visual Question Answering

Object Detection Based on Deep Learning

3D model classification using convolutional neural network

3D Shape Analysis with Multi-view Convolutional Networks. Evangelos Kalogerakis

Cross-domain Deep Encoding for 3D Voxels and 2D Images

Overview of 3D Object Representations

Scanning and Printing Objects in 3D Jürgen Sturm

POINT CLOUD DEEP LEARNING

From 3D descriptors to monocular 6D pose: what have we learned?

Feature Visualization

arxiv: v1 [cs.cv] 13 Feb 2018

Recognizing people. Deva Ramanan

arxiv: v1 [cs.cv] 31 Mar 2016

Learning Probabilistic Models from Collections of 3D Meshes

Recurrent Transformer Networks for Semantic Correspondence

Deep Models for 3D Reconstruction

ECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016

Multi-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011

Thanks to Chris Bregler. COS 429: Computer Vision

Collaborative Mapping with Streetlevel Images in the Wild. Yubin Kuang Co-founder and Computer Vision Lead

CS 395T Numerical Optimization for Graphics and AI (3D Vision) Qixing Huang August 29 th 2018

Learning Semantic Video Captioning using Data Generated with Grand Theft Auto

arxiv: v1 [cs.cv] 31 Mar 2018

Finding Surface Correspondences With Shape Analysis

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Supplementary Material

Three-Dimensional Object Detection and Layout Prediction using Clouds of Oriented Gradients

arxiv: v3 [cs.cv] 31 Oct 2017

SHREC 18: RGB-D Object-to-CAD Retrieval

FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans

arxiv: v1 [cs.cv] 23 Nov 2017

Is Bigger CNN Better? Samer Hijazi on behalf of IPG CTO Group Embedded Neural Networks Summit (enns2016) San Jose Feb. 9th

Linking WordNet to 3D Shapes

LEARNING TO GENERATE CHAIRS WITH CONVOLUTIONAL NEURAL NETWORKS

Large-Scale Point Cloud Classification Benchmark

arxiv: v1 [cs.cv] 30 Sep 2018

Deep 3D Machine Learning for Reconstruction and Repair of 3D Surfaces

COMPARATIVE DEEP LEARNING FOR CONTENT- BASED MEDICAL IMAGE RETRIEVAL

Video Object Segmentation using Deep Learning

3 Object Detection. BVM 2018 Tutorial: Advanced Deep Learning Methods. Paul F. Jaeger, Division of Medical Image Computing

LEARNING RIGIDITY IN DYNAMIC SCENES FOR SCENE FLOW ESTIMATION

Depth from Stereo. Dominic Cheng February 7, 2018

Martian lava field, NASA, Wikipedia

Deep Incremental Scene Understanding. Federico Tombari & Christian Rupprecht Technical University of Munich, Germany

Object Classification in Domestic Environments

Action recognition in robot-assisted minimally invasive surgery

Multi-view stereo. Many slides adapted from S. Seitz

Lecture 2 Notes. Outline. Neural Networks. The Big Idea. Architecture. Instructors: Parth Shah, Riju Pahwa

Todo before next class

Deconvolution Networks

3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

arxiv: v1 [cs.cv] 25 Oct 2017

GAL: Geometric Adversarial Loss for Single-View 3D-Object Reconstruction

VISION FOR AUTOMOTIVE DRIVING

arxiv: v1 [cs.cv] 2 Nov 2018

Depth-aware CNN for RGB-D Segmentation

Transcription:

Visual Computing Group @ TUM

Visual Computing Group @ TUM

BundleFusion Real-time 3D Reconstruction Scalable scene representation Global alignment and re-localization TOG 17 [Dai et al.]: BundleFusion

Real-time 3D Reconstruction TOG 17 [Dai et al.]: BundleFusion

Understanding the 3D Scans chair? chair? chair? chair? chair? TOG 17 [Dai et al.]: BundleFusion

3D Object Classification 3D Convolutional Neural Network operating on Voxel Grid CVPR 16 [Qi et al.]: 3D CNNs

3D Object Classification Synthetic Data Real Data CVPR 16 [Qi et al.]: 3D CNNs

ScanNet: Annotated 3D Reconstructions CVPR 17 (spotlight) [Dai et al.]: ScanNet

ScanNet Data Annotation CVPR 17 (spotlight) [Dai et al.]: ScanNet

ScanNet CVPR 17 (spotlight) [Dai et al.]: ScanNet

ScanNet Statistics CVPR 17 (spotlight) [Dai et al.]: ScanNet

CVPR 17 (spotlight) [Dai et al.]: ScanNet ScanNet Tasks: 3D Object Classification Synthetic Prev. Real Ours Lots of real-world data matters!

ScanNet Tasks 3D Object Classification 3D Semantic Segmentation 3D Instance Segmentation

ScanNet 2D Projections Color Depth Labels 2.5 mio annotated images for 2D object classification, semantic, and instance segmentation CVPR 17 (spotlight) [Dai et al.]: ScanNet

ScanNet v1 Problems Annotations are incomplete Test set is public -> people cheat

ScanNet v2: 3D Scene Understanding Benchmark New annotations for all 1513 original scans (by experts)!

ScanNet v2: 3D Scene Understanding Benchmark New annotations for all 1513 original scans (by experts)! 90% annotation coverage vs previous 63% annotation coverage

ScanNet v2: 3D Scene Understanding Benchmark New annotations for all 1513 original scans (by experts)! 90% annotation coverage vs previous 63% annotation coverage Annotated spurious geometry removed from meshes New room type classifications for all 1513 original scans (24 room types) New hidden test set for benchmarking (100 scans) Benchmark Tasks! Multi-view 2d 3D (+optional multi-view) Check it out http://www.scan-net.org/

ScanNet v2: 3D Semantic Segmentation Task http://www.scan-net.org/

ScanNet v2: 3D Instance Segmentation Task http://www.scan-net.org/

ScanNet v2: Multi-view 2D Semantic Segmentation Task http://www.scan-net.org/

ScanNet v2: Multi-view 2D Instance Segmentation Task http://www.scan-net.org/

ScanNet v2: Room Type Classification Task Room type classification ( 13 major room types) 2d and/or 3d input Apartment http://www.scan-net.org/

ScanNet v2: Benchmark Evaluation Evaluation in 2D evaluation over image pixels Evaluation in 3D evaluation over mesh vertices Hidden test sets and automated evaluation scripts! http://www.scan-net.org/

ScanNet v2: Benchmark Evaluation Online since CVPR http://www.scan-net.org/

ScanNet v2: Benchmark Evaluation

ScanNet v2: Benchmark Evaluation

ScanNet v2: Benchmark Evaluation

ScanNet v2: Benchmark Evaluation

3D Semantic Segmentation

3D Semantic Segmentation CVPR 17 (spotlight) [Dai et al.]: ScanNet

CVPR 17 (spotlight) [Dai et al.]: ScanNet ScanNet Tasks: 3D Semantic Segmentation Semantic 3D Segmentation

ScanNet Tasks: 3D Semantic Segmentation CVPR 17 (spotlight) [Dai et al.]: ScanNet

3DMV: Semantic 3D Segmentation ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: Semantic 3D Segmentation Under submission [Dai and Niessner 18]: 3DMV

3DMV: Semantic 3D Segmentation ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: 2D -> 3D Projections ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: 2D Input -> 3D Convs ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: Geo only vs Voxel Colors ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: Geometry vs Color Features ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: Geometry vs Color Features ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: More Views Helps ECCV 18 [Dai and Niessner 18]: 3DMV

3DMV: Semantic 3D Segmentation ECCV 18 [Dai and Niessner 18]: 3DMV

3D Scene Understanding + Combination of Multi-view + 3D works extremely well + Great way to combine multiple frames (best perf. so far) - Instances are missing (ongoing work ) - Still interesting question: what is right 3D representation

Thank You ScanNet Team Angela Dai Angel Chang Thomas Funkhouser Maciej Halber Manolis Savva http://niessnerlab.org

2024 © technodocbox.com Privacy Policy | Terms of Service | Feedback