3D Design with High- Level, Data-Driven Priors

Size: px
Start display at page:

Download "3D Design with High- Level, Data-Driven Priors"

Transcription

1 3D Design with High- Level, Data-Driven Priors Siddhartha Chaudhuri Assistant Professor, Dept of Computer Science & Engineering, IIT Bombay

2 Shapes are everywhere! Ake Axelsson

3 Shapes are everywhere! 3dhousedownload.com

4 Shapes are everywhere! MakerBot Industries

5 Shapes are everywhere! AFRL Discovery Lab/Aurora Flight Sciences

6 Shapes are everywhere! Saxena Lab, Cornell/Stanford

7 Shapes are everywhere! Google Inc.

8 Autodesk

9

10

11 rollins.edu

12 How can we create more widely usable design tools Humans give high-level directions Specified in terms of shape semantics The meaning/purpose/function of the design Make me a comfortable chair Computers handle low-level details Geometry, assembly, optimization

13 Design as Optimization

14 Design as Optimization

15 Design as Optimization

16 Design as Optimization

17 Two Big Questions How can we identify the plausible regions of shape space (Probabilistic) models of shape structure How can we identify and operationalize design intent Computational models of attributes, interaction and function Writing these models by hand is tedious and error-prone Statistical (machine) learning: Show examples, let the algorithm regress the underlying pattern

18 Generative Structure SIGAsia 2010 SIGGRAPH 2011 SIGGRAPH 2012 SIGGRAPH 2017 SIGAsia 2018 TOG 2018/19 SIGAsia 2017 Intent and Function Geometric Analysis UIST 2013 SIGGRAPH 2014 SIGGRAPH 2015 CVPR 2018 SIGGRAPH 2013 SIGAsia 2014 SGP 2016 CVPR 2017 TOG DV 2018

19 Generative Structure Projective convolutional neural networks SIGAsia 2010 SIGGRAPH 2011 SIGGRAPH 2012 SIGGRAPH 2017 SIGAsia 2018 TOG 2018/19 SIGAsia 2017 Intent and Function Geometric Analysis UIST 2013 SIGGRAPH 2014 SIGGRAPH 2015 CVPR 2018 SIGGRAPH 2013 SIGAsia 2014 SGP 2016 CVPR 2017 TOG DV 2018

20 Generative Structure SIGAsia 2010 SIGGRAPH 2011 SIGGRAPH 2012 SIGGRAPH 2017 SIGAsia 2018 TOG 2018/19 SIGAsia 2017 Intent and Function UIST 2013 SIGGRAPH 2014 SIGGRAPH 2015 CVPR 2018 Geometric Analysis Weakly-supervised deep learning SIGGRAPH 2013 SIGAsia 2014 SGP 2016 CVPR 2017 TOG DV 2018

21 Exemplar 3D Shapes Google/Trimble 3D Warehouse (~millions of downloadable models)

22 Probabilistic Part-Based Shape Model Shape style Z + Number of parts from a category {0} Z + Part style {0} Z + Continuous feature vector R n Discrete feature vector Z m Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

23 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

24 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

25 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

26 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

27 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

28 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

29 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

30 Shape Synthesis Kalogerakis, Chaudhuri, Koller and Koltun, SIGGRAPH 2012

31 Simple model captures limited variations Shapes can have arbitrary numbers/types of nodes (parts), arbitrary numbers of connections (adjacencies/symmetries) Wikipedia

32 Key Insight Edges of a graph can be collapsed sequentially into a hierarchical structure Recursive Neural Network (RvNN): Repeatedly merge two nodes into one Each node has an n-d feature vector, computed recursively p = f (W [c 1 ;c 2 ] + b) Back-prop the entire dynamic structure

33 GRASS: Synthesize and interpolate shapes (with topology changes) Variational Autoencoder Before Real structures Generative Adversarial Network After Li, Xu, Chaudhuri, Yumer, Zhang, Guibas, SIGGRAPH 2017

34 SCORES: Shape composition with substructure priors (with missing and redundant parts) + = Composition Evolution Reconstruction Zhu, Xu, Chaudhuri, Yi, Zhang, SIGGRAPH Asia 2018

35 GRAINS: Recursive scene synthesis (new scene every feedforward pass, with a scene hierarchy for free) Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or, Zhang, TOG 2018/19

36 Semantic Basis for Shape Space x 2 x 1

37 Semantic Basis for Shape Space x 2 Scary Strong x 1

38 Semantic Basis for Shape Space x 2 Scary Strong x 1

39 Semantic Basis for Shape Space x 2 Scary Strong x 1

40 A cute toy for a small child Chaudhuri, Kalogerakis, Giguere and Funkhouser, UIST 2013

41 Continuous semantic editing Yumer, Chaudhuri, Hodgins and Kara, SIGGRAPH 2015

42 Thanks! Rajendra Adiga, Melinos Averkiou, Kavita Bala, Sean Bell, Vishal Bhavani, Sourav Bose, Duygu Ceylan, Soumen Chakrabarti, Abhishek Chakraborty, Parag Chaudhuri, Subhasis Chaudhuri, Baoquan Chen, Daniel Cohen-Or. Stephen DiVerdi, Thomas Funkhouser, Srajan Garg, Sandip Ghoshal, Steve Giguere, Leonidas Guibas, Xuekun Guo, Jessica Hodgins, Haibin Huang, Qixing Huang, Xiaogang Jin, Preethi Jyothi, Evangelos Kalogerakis, Shivaram Kalyanakrishnan, L. Burak Kara, Owais Khan, Vladimir Kim, Honza Knopp, Daphne Koller, Vladlen Koltun, Balazs Kovacs, Priyadarshini Kumari, Uday Kusupati, G. Roshan Lal, Jun Li, Manyi Li, Wilmot Li, Hubert Lin, Juncong Lin, Tianqiang Liu, Subhransu Maji, Utkarsh Mall, Niloy Mitra, Kaichun Mo, Sanjeev Mk, Srinath Naik, Akshay Gadi Patil, Vihari Piratla, Vivek Punia, Siddhant Ranade, Sunita Sarawagi, Maulik Shah, Ariel Shamir, Shiv Shankar, Nitish Sontakke, Hao Su, Minhyuk Sung, Changhe Tu, Nikhil Vyas, TongKe Xue, Kai Xu, Pushyarag Y, Renjiao Yi, Ersin Yumer, Hao (Richard) Zhang, Chenyang Zhu, the ViGIL lab, Dept of CSE, IIT Bombay, and all my students in CS749 and CS475 Funding Agencies: IIT Bombay, National Science Foundation, Intel, Adobe, Qualcomm, PACCAR

Learning Probabilistic Models from Collections of 3D Meshes

Learning Probabilistic Models from Collections of 3D Meshes Learning Probabilistic Models from Collections of 3D Meshes Sid Chaudhuri, Steve Diverdi, Matthew Fisher, Pat Hanrahan, Vladimir Kim, Wilmot Li, Niloy Mitra, Daniel Ritchie, Manolis Savva, and Thomas Funkhouser

More information

Discovering Similarities in 3D Data

Discovering Similarities in 3D Data Discovering Similarities in 3D Data Vladimir Kim, Tianqiang Liu, Sid Chaudhuri, Steve Diverdi, Wilmot Li, Niloy Mitra, Yaron Lipman, Thomas Funkhouser Motivation 3D data is widely available Medicine Mechanical

More information

Machine Learning for Shape Analysis and Processing. Evangelos Kalogerakis

Machine Learning for Shape Analysis and Processing. Evangelos Kalogerakis Machine Learning for Shape Analysis and Processing Evangelos Kalogerakis 3D shapes for computer aided design Architecture Interior design 3D shapes for information visualization Geo visualization Scientific

More information

Machine Learning Techniques for Geometric Modeling. Evangelos Kalogerakis

Machine Learning Techniques for Geometric Modeling. Evangelos Kalogerakis Machine Learning Techniques for Geometric Modeling Evangelos Kalogerakis 3D models for digital entertainment Limit Theory 3D models for printing MakerBot Industries 3D models for architecture Architect:

More information

arxiv: v4 [cs.cv] 29 Mar 2019

arxiv: v4 [cs.cv] 29 Mar 2019 PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation Fenggen Yu 1 Kun Liu 1 * Yan Zhang 1 Chenyang Zhu 2 Kai Xu 2 1 Nanjing University 2 National University

More information

Depth Estimation Using Collection of Models

Depth Estimation Using Collection of Models Depth Estimation Using Collection of Models Karankumar Thakkar 1, Viral Borisagar 2 PG scholar 1, Assistant Professor 2 Computer Science & Engineering Department 1, 2 Government Engineering College, Sector

More information

arxiv: v2 [cs.gr] 13 Sep 2017

arxiv: v2 [cs.gr] 13 Sep 2017 ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling arxiv:1708.01841v2 [cs.gr] 13 Sep 2017 MINHYUK SUNG, Stanford University HAO SU, Stanford University and University of California San

More information

GRAINS: Generative Recursive Autoencoders for INdoor Scenes

GRAINS: Generative Recursive Autoencoders for INdoor Scenes GRAINS: Generative Recursive Autoencoders for INdoor Scenes MANYI LI, Shandong University and Simon Fraser University AKSHAY GADI PATIL, Simon Fraser University KAI XU, National University of Defense Technology

More information

Finding Structure in Large Collections of 3D Models

Finding Structure in Large Collections of 3D Models Finding Structure in Large Collections of 3D Models Vladimir Kim Adobe Research Motivation Explore, Analyze, and Create Geometric Data Real Virtual Motivation Explore, Analyze, and Create Geometric Data

More information

3D Shape Analysis with Multi-view Convolutional Networks. Evangelos Kalogerakis

3D Shape Analysis with Multi-view Convolutional Networks. Evangelos Kalogerakis 3D Shape Analysis with Multi-view Convolutional Networks Evangelos Kalogerakis 3D model repositories [3D Warehouse - video] 3D geometry acquisition [KinectFusion - video] 3D shapes come in various flavors

More information

Exploring Collections of 3D Models using Fuzzy Correspondences

Exploring Collections of 3D Models using Fuzzy Correspondences Exploring Collections of 3D Models using Fuzzy Correspondences Vladimir G. Kim Wilmot Li Niloy J. Mitra Princeton University Adobe UCL Stephen DiVerdi Adobe Thomas Funkhouser Princeton University Motivating

More information

Finding Surface Correspondences With Shape Analysis

Finding Surface Correspondences With Shape Analysis Finding Surface Correspondences With Shape Analysis Sid Chaudhuri, Steve Diverdi, Maciej Halber, Vladimir Kim, Yaron Lipman, Tianqiang Liu, Wilmot Li, Niloy Mitra, Elena Sizikova, Thomas Funkhouser Motivation

More information

Analysis and Synthesis of 3D Shape Families via Deep Learned Generative Models of Surfaces

Analysis and Synthesis of 3D Shape Families via Deep Learned Generative Models of Surfaces Analysis and Synthesis of 3D Shape Families via Deep Learned Generative Models of Surfaces Haibin Huang, Evangelos Kalogerakis, Benjamin Marlin University of Massachusetts Amherst Given an input 3D shape

More information

3D Deep Learning

3D Deep Learning 3D Deep Learning Tutorial@CVPR2017 Hao Su (UCSD) Leonidas Guibas (Stanford) Michael Bronstein (Università della Svizzera Italiana) Evangelos Kalogerakis (UMass) Jimei Yang (Adobe Research) Charles Qi (Stanford)

More information

Data driven 3D shape analysis and synthesis

Data driven 3D shape analysis and synthesis Data driven 3D shape analysis and synthesis Head Neck Torso Leg Tail Ear Evangelos Kalogerakis UMass Amherst 3D shapes for computer aided design Architecture Interior design 3D shapes for information visualization

More information

3D Shape Segmentation with Projective Convolutional Networks

3D Shape Segmentation with Projective Convolutional Networks 3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis 1 Melinos Averkiou 2 Subhransu Maji 1 Siddhartha Chaudhuri 3 1 University of Massachusetts Amherst 2 University of Cyprus

More information

Learning Photographic Image Synthesis With Cascaded Refinement Networks. Jonathan Louie Huy Doan Siavash Motalebi

Learning Photographic Image Synthesis With Cascaded Refinement Networks. Jonathan Louie Huy Doan Siavash Motalebi Learning Photographic Image Synthesis With Cascaded Refinement Networks Jonathan Louie Huy Doan Siavash Motalebi Introduction and Background Intro We are researching and re-implementing Photographic Image

More information

Modeling and Analyzing 3D Shapes using Clues from 2D Images. Minglun Gong Dept. of CS, Memorial Univ.

Modeling and Analyzing 3D Shapes using Clues from 2D Images. Minglun Gong Dept. of CS, Memorial Univ. Modeling and Analyzing 3D Shapes using Clues from 2D Images Minglun Gong Dept. of CS, Memorial Univ. Modeling Flowers from Images Reconstructing 3D flower petal shapes from a single image is difficult

More information

Data-Driven Geometry Processing Map Synchronization I. Qixing Huang Nov. 28 th 2018

Data-Driven Geometry Processing Map Synchronization I. Qixing Huang Nov. 28 th 2018 Data-Driven Geometry Processing Map Synchronization I Qixing Huang Nov. 28 th 2018 Shape matching Affine Applications Shape reconstruction Transfer information Aggregate information Protein docking Pair-wise

More information

Learning from 3D Data

Learning from 3D Data Learning from 3D Data Thomas Funkhouser Princeton University* * On sabbatical at Stanford and Google Disclaimer: I am talking about the work of these people Shuran Song Andy Zeng Fisher Yu Yinda Zhang

More information

Shape Co-analysis. Daniel Cohen-Or. Tel-Aviv University

Shape Co-analysis. Daniel Cohen-Or. Tel-Aviv University Shape Co-analysis Daniel Cohen-Or Tel-Aviv University 1 High-level Shape analysis [Fu et al. 08] Upright orientation [Mehra et al. 08] Shape abstraction [Kalograkis et al. 10] Learning segmentation [Mitra

More information

Brief Biography. Personal Data Name: Leonidas J. Guibas Address: Computer Science Department. Stanford University

Brief Biography. Personal Data Name: Leonidas J. Guibas Address: Computer Science Department. Stanford University Brief Biography Personal Data Name: Leonidas J. Guibas Address: Computer Science Department Stanford, CA 94305 Tel.: (650) 723-0304 E-mail: guibas@cs.stanford.edu Website: http://geometry.stanford.edu

More information

Global-to-Local Generative Model for 3D Shapes

Global-to-Local Generative Model for 3D Shapes Global-to-Local Generative Model for 3D Shapes HAO WANG,ShenzhenUniversity NADAVSCHOR, Tel AvivUniversity RUIZHEN HU, Shenzhen University HAIBIN HUANG, Megvii/ Face++ Research DANIEL COHEN-OR, Shenzhen

More information

Recurring Part Arrangements in Shape Collections

Recurring Part Arrangements in Shape Collections EUROGRAPHICS 2014 / B. Lévy and J. Kautz (Guest Editors) Volume 33 (2014), Number 2 Recurring Part Arrangements in Shape Collections Youyi Zheng 1 Daniel Cohen-Or 2 Melinos Averkiou 3 Niloy J. Mitra 3,1

More information

Learning to generate 3D shapes

Learning to generate 3D shapes Learning to generate 3D shapes Subhransu Maji College of Information and Computer Sciences University of Massachusetts, Amherst http://people.cs.umass.edu/smaji August 10, 2018 @ Caltech Creating 3D shapes

More information

Structure-oriented Networks of Shape Collections

Structure-oriented Networks of Shape Collections Structure-oriented Networks of Shape Collections Noa Fish 1 Oliver van Kaick 2 Amit Bermano 3 Daniel Cohen-Or 1 1 Tel Aviv University 2 Carleton University 3 Princeton University 1 pplementary material

More information

Learning Adversarial 3D Model Generation with 2D Image Enhancer

Learning Adversarial 3D Model Generation with 2D Image Enhancer The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) Learning Adversarial 3D Model Generation with 2D Image Enhancer Jing Zhu, Jin Xie, Yi Fang NYU Multimedia and Visual Computing Lab

More information

Seeing the unseen. Data-driven 3D Understanding from Single Images. Hao Su

Seeing the unseen. Data-driven 3D Understanding from Single Images. Hao Su Seeing the unseen Data-driven 3D Understanding from Single Images Hao Su Image world Shape world 3D perception from a single image Monocular vision a typical prey a typical predator Cited from https://en.wikipedia.org/wiki/binocular_vision

More information

Photo-realistic Renderings for Machines Seong-heum Kim

Photo-realistic Renderings for Machines Seong-heum Kim Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,

More information

3D Attention-Driven Depth Acquisition for Object Identification

3D Attention-Driven Depth Acquisition for Object Identification 3D Attention-Driven Depth Acquisition for Object Identification Kai Xu, Yifei Shi, Lintao Zheng, Junyu Zhang, Min Liu, Hui Huang, Hao Su, Daniel Cohen-Or and Baoquan Chen National University of Defense

More information

3D Scene Understanding from RGB-D Images. Thomas Funkhouser

3D Scene Understanding from RGB-D Images. Thomas Funkhouser 3D Scene Understanding from RGB-D Images Thomas Funkhouser Recent Ph.D. Student Current Postdocs Current Ph.D. Students Disclaimer: I am talking about the work of these people Shuran Song Yinda Zhang Andy

More information

arxiv: v1 [cs.cv] 30 Sep 2018

arxiv: v1 [cs.cv] 30 Sep 2018 3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction From a Single Image Priyanka Mandikal, Navaneet K L, and R. Venkatesh Babu arxiv:1810.00461v1 [cs.cv] 30 Sep 2018 Indian Institute of Science, Bangalore,

More information

arxiv: v1 [cs.gr] 22 Sep 2017

arxiv: v1 [cs.gr] 22 Sep 2017 Hierarchical Detail Enhancing Mesh-Based Shape Generation with 3D Generative Adversarial Network arxiv:1709.07581v1 [cs.gr] 22 Sep 2017 Chiyu Max Jiang University of California Berkeley, CA 94720 chiyu.jiang@berkeley.edu

More information

Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling

Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story

More information

arxiv: v2 [cs.cv] 4 Jan 2017

arxiv: v2 [cs.cv] 4 Jan 2017 Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling Jiajun Wu* Chengkai Zhang* Tianfan Xue MIT CSAIL MIT CSAIL MIT CSAIL arxiv:1610.07584v2 [cs.cv] 4 Jan 2017

More information

3D Deep Learning on Geometric Forms. Hao Su

3D Deep Learning on Geometric Forms. Hao Su 3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates: multi-view images depth map volumetric polygonal mesh point cloud primitive-based CAD models 3D representation

More information

Creating Consistent Scene Graphs Using a Probabilistic Grammar

Creating Consistent Scene Graphs Using a Probabilistic Grammar Creating Consistent Scene Graphs Using a Probabilistic Grammar Tianqiang Liu 1 Siddhartha Chaudhuri 1,2 Vladimir G. Kim 3 Qixing Huang 3,4 Niloy J. Mitra 5 Thomas Funkhouser 1 1 Princeton University 2

More information

Data-Driven Shape Analysis --- Joint Shape Matching I. Qi-xing Huang Stanford University

Data-Driven Shape Analysis --- Joint Shape Matching I. Qi-xing Huang Stanford University Data-Driven Shape Analysis --- Joint Shape Matching I Qi-xing Huang Stanford University 1 Shape matching Affine Applications Shape reconstruction Transfer information Aggregate information Protein docking

More information

Lab meeting (Paper review session) Stacked Generative Adversarial Networks

Lab meeting (Paper review session) Stacked Generative Adversarial Networks Lab meeting (Paper review session) Stacked Generative Adversarial Networks 2017. 02. 01. Saehoon Kim (Ph. D. candidate) Machine Learning Group Papers to be covered Stacked Generative Adversarial Networks

More information

AMID BASED CROWD DENSITY ESTIMATION

AMID BASED CROWD DENSITY ESTIMATION AMID BASED CROWD DENSITY ESTIMATION Rupali Patil 1, Yuvaraj Patil 2 1M.E student, Dept.of Electronics Engineering, KIT s College of Engineering, Maharastra, India 2Professor Dept.of Electronics Engineering,

More information

Towards 3D Neural Style Transfer

Towards 3D Neural Style Transfer Towards 3D Neural Style Transfer Jo Mazeika, Jim Whitehead Computer Science Department UC Santa Cruz Santa Cruz, CA 95064 USA {jmazeika, ejw}@soe.ucsc.edu Abstract Neural Style Transfer was first unveiled

More information

3D Printing Oriented Design: Geometry and Optimization

3D Printing Oriented Design: Geometry and Optimization 3D Printing Oriented Design: Geometry and Optimization Siggraph Asia 2014 Course Dec. 5, 2014, Shenzhen Ligang Liu, Charlie Wang, Ariel Shamir, Emily Whiting Web: http://staff.ustc.edu.cn/~lgliu/courses/sigasia_2014_course_3dprinting/index.html

More information

3D Shape Segmentation with Projective Convolutional Networks

3D Shape Segmentation with Projective Convolutional Networks 3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis 1 Melinos Averkiou 2 Subhransu Maji 1 Siddhartha Chaudhuri 3 1 University of Massachusetts Amherst 2 University of Cyprus

More information

ECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016

ECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016 ECCV 2016 Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016 Fundamental Question What is a good vector representation of an object? Something that can be easily predicted from 2D

More information

Modeling Interactive Furniture from a Single Image

Modeling Interactive Furniture from a Single Image Modeling Interactive Furniture from a Single Image Bo Gao, Xiaowu Chen, Jianwei Li, Dongqing Zou State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering,

More information

arxiv:submit/ [cs.cv] 23 Aug 2018

arxiv:submit/ [cs.cv] 23 Aug 2018 Parsing Geometry Using Structure-Aware Shape Templates arxiv:submit/2372951 [cs.cv] 23 Aug 2018 Vignesh Ganapathi-Subramanian Stanford University vigansub@stanford.edu Chengcheng Tang Stanford University

More information

LEARNING TO DESCRIBE SCENES WITH PROGRAMS

LEARNING TO DESCRIBE SCENES WITH PROGRAMS LEARNING TO DESCRIBE SCENES WITH PROGRAMS Yunchao Liu IIIS, Tsinghua University Zheng Wu MIT CSAIL, Shanghai Jiao Tong University Daniel Ritchie Brown University William T. Freeman MIT CSAIL, Google Research

More information

Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material

Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Charles R. Qi Hao Su Matthias Nießner Angela Dai Mengyuan Yan Leonidas J. Guibas Stanford University 1. Details

More information

Generative Networks. James Hays Computer Vision

Generative Networks. James Hays Computer Vision Generative Networks James Hays Computer Vision Interesting Illusion: Ames Window https://www.youtube.com/watch?v=ahjqe8eukhc https://en.wikipedia.org/wiki/ames_trapezoid Recap Unsupervised Learning Style

More information

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space Sikai Zhong February 14, 2018 COMPUTER SCIENCE Table of contents 1. PointNet 2. PointNet++ 3. Experiments 1 PointNet Property

More information

arxiv: v1 [cs.gr] 4 May 2017

arxiv: v1 [cs.gr] 4 May 2017 Learning Hierarchical Shape Segmentation and Labeling from Online Repositories Li Yi 1 Leonidas Guibas 1 Aaron Hertzmann 2 Vladimir G. Kim 2 Hao Su 1 Ersin Yumer 2 1 Stanford University 2 Adobe Research

More information

arxiv: v1 [cs.cv] 2 Nov 2018

arxiv: v1 [cs.cv] 2 Nov 2018 3D Pick & Mix: Object Part Blending in Joint Shape and Image Manifolds Adrian Penate-Sanchez 1,2[0000 0003 2876 3301] Lourdes Agapito 1[0000 0002 6947 1092] arxiv:1811.01068v1 [cs.cv] 2 Nov 2018 1 University

More information

Classifying a specific image region using convolutional nets with an ROI mask as input

Classifying a specific image region using convolutional nets with an ROI mask as input Classifying a specific image region using convolutional nets with an ROI mask as input 1 Sagi Eppel Abstract Convolutional neural nets (CNN) are the leading computer vision method for classifying images.

More information

arxiv: v1 [cs.gr] 13 Sep 2018

arxiv: v1 [cs.gr] 13 Sep 2018 Learning to Group and Label Fine-Grained Shape Components XIAOGANG WANG, Beihang University BIN ZHOU, Beihang University HAIYUE FANG, Beihang University XIAOWU CHEN, Beihang University QINPING ZHAO, Beihang

More information

ExquiMo: An Exquisite Corpse Tool for Co-creative 3D Shape Modeling

ExquiMo: An Exquisite Corpse Tool for Co-creative 3D Shape Modeling ExquiMo: An Exquisite Corpse Tool for Co-creative 3D Shape Modeling Abstract We introduce a shape modeling tool, ExquiMo, which is guided by the idea of improving the creativity of 3D shape designs through

More information

The Future of Natural User Interaction

The Future of Natural User Interaction The Future of Natural User Interaction NUI: a great new discipline in the making Baining Guo Microsoft Research Asia Kinect from Insiders Perspective An interview with Kinect contributors Kinect from

More information

arxiv: v1 [cs.cv] 12 Apr 2019

arxiv: v1 [cs.cv] 12 Apr 2019 Learning Shape Templates with Structured Implicit Functions Kyle Genova 1,2 Forrester Cole 2 Daniel Vlasic 2 Aaron Sarna 2 William T. Freeman 2 Thomas Funkhouser 1, 2 1 Princeton University 2 Google Research

More information

arxiv: v3 [cs.cv] 13 Nov 2017

arxiv: v3 [cs.cv] 13 Nov 2017 3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis 1 Melinos Averkiou 2 Subhransu Maji 1 Siddhartha Chaudhuri 3 1 University of Massachusetts Amherst 2 University of Cyprus

More information

S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T

S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T S U N G - E U I YO O N, K A I S T R E N D E R I N G F R E E LY A VA I L A B L E O N T H E I N T E R N E T Copyright 2018 Sung-eui Yoon, KAIST freely available on the internet http://sglab.kaist.ac.kr/~sungeui/render

More information

Sketch2Scene: Sketch-based Co-retrieval and Co-placement of 3D Models

Sketch2Scene: Sketch-based Co-retrieval and Co-placement of 3D Models Sketch2Scene: Sketch-based Co-retrieval and Co-placement of 3D Models Kun Xu 1 Kang Chen 1 Hongbo Fu 2 Wei-Lun Sun 1 Shi-Min Hu 1 1 Tsinghua University, Beijing 2 City University of Hong Kong Figure 1:

More information

arxiv: v3 [cs.cv] 9 Sep 2016

arxiv: v3 [cs.cv] 9 Sep 2016 arxiv:1604.03755v3 [cs.cv] 9 Sep 2016 VConv-DAE: Deep Volumetric Shape Learning Without Object Labels Abhishek Sharma 1, Oliver Grau 2, Mario Fritz 3 1 Intel Visual Computing Institute 2 Intel 3 Max Planck

More information

VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN

VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN VIDEO FOR VIRTUAL REALITY LIGHT FIELD BASICS JAMES TOMPKIN WHAT IS A LIGHT FIELD? Light field seems to have turned into a catch-all term for many advanced camera/display technologies. WHAT IS A LIGHT FIELD?

More information

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image. Supplementary Material

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image. Supplementary Material Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image Supplementary Material Siyuan Huang 1,2, Siyuan Qi 1,2, Yixin Zhu 1,2, Yinxue Xiao 1, Yuanlu Xu 1,2, and Song-Chun Zhu 1,2 1 University

More information

arxiv: v1 [cs.cv] 1 Dec 2017

arxiv: v1 [cs.cv] 1 Dec 2017 Multi-Content GAN for Few-Shot Font Style Transfer Samaneh Azadi 1, Matthew Fisher 2, Vladimir Kim 2, Zhaowen Wang 2, Eli Shechtman 2, Trevor Darrell 1 1 UC Berkeley, 2 Adobe Research {sazadi,trevor}@eecs.berkeley.edu

More information

An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks

An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks An Empirical Study of Generative Adversarial Networks for Computer Vision Tasks Report for Undergraduate Project - CS396A Vinayak Tantia (Roll No: 14805) Guide: Prof Gaurav Sharma CSE, IIT Kanpur, India

More information

Co-Hierarchical Analysis of Shape Structures

Co-Hierarchical Analysis of Shape Structures Co-Hierarchical Analysis of Shape Structures Oliver van Kaick Kai Xu Hao Zhang Yanzhen Wang Shuyang Sun Ariel Shamir Simon Fraser Univ. HPCL, Nat. Univ. of Defense Tech. Interdisciplinary Center Daniel

More information

Web-Scale Image Search and Their Applications

Web-Scale Image Search and Their Applications Web-Scale Image Search and Their Applications Sung-Eui Yoon KAIST http://sglab.kaist.ac.kr Project Guidelines: Project Topics Any topics related to the course theme are okay You can find topics by browsing

More information

Visual Computing TUM

Visual Computing TUM Visual Computing Group @ TUM Visual Computing Group @ TUM BundleFusion Real-time 3D Reconstruction Scalable scene representation Global alignment and re-localization TOG 17 [Dai et al.]: BundleFusion Real-time

More information

Deep 3D Machine Learning for Reconstruction and Repair of 3D Surfaces

Deep 3D Machine Learning for Reconstruction and Repair of 3D Surfaces Deep 3D Machine Learning for Reconstruction and Repair of 3D Surfaces TalkID 23152 This session will give the audience a quick overview of recent developments in the field of 3D surface analysis with deep

More information

CS354 Computer Graphics Introduction. Qixing Huang Januray 17 8h 2017

CS354 Computer Graphics Introduction. Qixing Huang Januray 17 8h 2017 CS354 Computer Graphics Introduction Qixing Huang Januray 17 8h 2017 CS 354 Computer Graphics Instructor: Qixing Huang huangqx@cs.utexas.edu Office: GDC 5.422 Office hours: Friday 3:00 pm 5:00 pm Teaching

More information

345 Park Ave,, San Jose, CA Homepage:

345 Park Ave,, San Jose, CA Homepage: Jimei Yang Contact Information Research Interests Working Experience 345 Park Ave,, San Jose, CA 95110 E-mail: jimyang@adobe.com Homepage: https://eng.ucmerced.edu/people/jyang44 Deep Learning, Computer

More information

3D model classification using convolutional neural network

3D model classification using convolutional neural network 3D model classification using convolutional neural network JunYoung Gwak Stanford jgwak@cs.stanford.edu Abstract Our goal is to classify 3D models directly using convolutional neural network. Most of existing

More information

Five Principles for Choosing Research Problems in Computer Graphics. Thomas Funkhouser Princeton University

Five Principles for Choosing Research Problems in Computer Graphics. Thomas Funkhouser Princeton University Five Principles for Choosing Research Problems in Computer Graphics Thomas Funkhouser Princeton University Why this Topic? Possible alternative topics I did this, then I did that, then. zzz What are the

More information

CS 395T Numerical Optimization for Graphics and AI (3D Vision) Qixing Huang August 29 th 2018

CS 395T Numerical Optimization for Graphics and AI (3D Vision) Qixing Huang August 29 th 2018 CS 395T Numerical Optimization for Graphics and AI (3D Vision) Qixing Huang August 29 th 2018 3D Vision Understanding geometric relations between images and the 3D world between images Obtaining 3D information

More information

AI Ukraine Point cloud labeling using machine learning

AI Ukraine Point cloud labeling using machine learning AI Ukraine - 2017 Point cloud labeling using machine learning Andrii Babii apratster@gmail.com Data sources 1. LIDAR 2. 3D IR cameras 3. Generated from 2D sources 3D dataset: http://kos.informatik.uni-osnabrueck.de/3dscans/

More information

GAN Related Works. CVPR 2018 & Selective Works in ICML and NIPS. Zhifei Zhang

GAN Related Works. CVPR 2018 & Selective Works in ICML and NIPS. Zhifei Zhang GAN Related Works CVPR 2018 & Selective Works in ICML and NIPS Zhifei Zhang Generative Adversarial Networks (GANs) 9/12/2018 2 Generative Adversarial Networks (GANs) Feedforward Backpropagation Real? z

More information

Unsupervised Learning

Unsupervised Learning Deep Learning for Graphics Unsupervised Learning Niloy Mitra Iasonas Kokkinos Paul Guerrero Vladimir Kim Kostas Rematas Tobias Ritschel UCL UCL/Facebook UCL Adobe Research U Washington UCL Timetable Niloy

More information

Recurrent Transformer Networks for Semantic Correspondence

Recurrent Transformer Networks for Semantic Correspondence Neural Information Processing Systems (NeurIPS) 2018 Recurrent Transformer Networks for Semantic Correspondence Seungryong Kim 1, Stepthen Lin 2, Sangryul Jeon 1, Dongbo Min 3, Kwanghoon Sohn 1 Dec. 05,

More information

Style Compatibility for 3D Furniture Models

Style Compatibility for 3D Furniture Models Style Compatibility for 3D Furniture Models Tianqiang Liu 1 Aaron Hertzmann 2 Wilmot Li 2 Thomas Funkhouser 1 1 Princeton University 2 Adobe Research (a) Ignoring style compatibility (b) Optimizing style

More information

Physical Primitive Decomposition

Physical Primitive Decomposition Physical Primitive Decomposition Zhijian Liu 1, William T. Freeman 1,2, Joshua B. Tenenbaum 1, and Jiajun Wu 1 1 Massachusetts Institute of Technology 2 Google Research Abstract. Objects are made of parts,

More information

arxiv: v4 [cs.cv] 2 Aug 2018

arxiv: v4 [cs.cv] 2 Aug 2018 Learning Shape Abstractions by Assembling Volumetric Primitives Shubham Tulsiani 1, Hao Su 2, Leonidas J. Guibas 2, Alexei A. Efros 1, Jitendra Malik 1 1 University of California, Berkeley 2 Stanford University

More information

Lecture 7: Semantic Segmentation

Lecture 7: Semantic Segmentation Semantic Segmentation CSED703R: Deep Learning for Visual Recognition (207F) Segmenting images based on its semantic notion Lecture 7: Semantic Segmentation Bohyung Han Computer Vision Lab. bhhanpostech.ac.kr

More information

Urban Layouts and Road Networks

Urban Layouts and Road Networks Urban Layouts and Road Networks Daniel G. Aliaga Associate Professor of Computer Science Purdue University Challenge Design and model realistic/plausible road networks and urban layouts Road network: a

More information

Interactive Design and Visualization of Urban Spaces using Geometrical and Behavioral Modeling

Interactive Design and Visualization of Urban Spaces using Geometrical and Behavioral Modeling Interactive Design and Visualization of Urban Spaces using Geometrical and Behavioral Modeling Carlos Vanegas 1,4,5 Daniel Aliaga 1 Bedřich Beneš 2 Paul Waddell 3 1 Computer Science, Purdue University,

More information

Feature Visualization

Feature Visualization CreativeAI: Deep Learning for Graphics Feature Visualization Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL UCL TU Munich UCL Timetable Theory and Basics State of the Art

More information

PROCEDURAL Modeling (PM) allows synthesis of complex

PROCEDURAL Modeling (PM) allows synthesis of complex IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2017, TO APPEAR 1 Shape Synthesis from Sketches via Procedural Models and Convolutional Networks Haibin Huang, Evangelos Kalogerakis, Ersin Yumer,

More information

Structure Recovery by Part Assembly

Structure Recovery by Part Assembly Structure Recovery by Part Assembly Chao-Hui Shen 1 Hongbo Fu 2 Kang Chen 1 Shi-Min Hu 1 1 TNList, Tsinghua University, Beijing 2 City University of Hong Kong Figure 1: Given single-view scans by the Kinect

More information

arxiv: v1 [cs.cv] 29 Mar 2018

arxiv: v1 [cs.cv] 29 Mar 2018 arxiv:1803.10932v1 [cs.cv] 29 Mar 2018 Learning Free-Form Deformations for 3D Object Reconstruction Dominic Jack, Jhony K. Pontes, Sridha Sridharan, Clinton Fookes, Sareh Shirazi, Frederic Maire, Anders

More information

Multi-Content GAN for Few-Shot Font Style Transfer

Multi-Content GAN for Few-Shot Font Style Transfer Multi-Content GAN for Few-Shot Font Style Transfer Samaneh Azadi 1, Matthew Fisher 2, Vladimir Kim 2, Zhaowen Wang 2, Eli Shechtman 2, Trevor Darrell 1 1 UC Berkeley, 2 Adobe Research {sazadi,trevor}@eecs.berkeley.edu

More information

Deep Generative Models and a Probabilistic Programming Library

Deep Generative Models and a Probabilistic Programming Library Deep Generative Models and a Probabilistic Programming Library Discriminative (Deep) Learning Learn a (differentiable) function mapping from input to output x f(x; θ) y Gradient back-propagation Generative

More information

LSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University

LSTM and its variants for visual recognition. Xiaodan Liang Sun Yat-sen University LSTM and its variants for visual recognition Xiaodan Liang xdliang328@gmail.com Sun Yat-sen University Outline Context Modelling with CNN LSTM and its Variants LSTM Architecture Variants Application in

More information

Model-Driven Indoor Scenes Modeling from a Single Image

Model-Driven Indoor Scenes Modeling from a Single Image Graphics Interface Conference 2015, 3 5 June, Halifax, Nova Scotia, Canada Model-Driven Indoor Scenes Modeling from a Single Image Zicheng Liu 1,2 Yan Zhang 1,2 Wentao Wu 1,2 Kai Liu 1,2 Zhengxing Sun

More information

Technical Papers November 2017 Bangkok Int'l Trade & Exhibition Centre (BITEC)

Technical Papers November 2017 Bangkok Int'l Trade & Exhibition Centre (BITEC) Session: Form, Furniture and Fashion Date/Time: 28 November 2017, 09:00am - 10:45am Position-Based Tensegrity Design Material-Minimizing Forms and Structures Reconfigurable Interlocking Furniture Fabricable

More information

Decomposing a Scene into Geometric and Semantically Consistent Regions

Decomposing a Scene into Geometric and Semantically Consistent Regions Decomposing a Scene into Geometric and Semantically Consistent Regions Stephen Gould sgould@stanford.edu Richard Fulton rafulton@cs.stanford.edu Daphne Koller koller@cs.stanford.edu IEEE International

More information

Detecting and Parsing of Visual Objects: Humans and Animals. Alan Yuille (UCLA)

Detecting and Parsing of Visual Objects: Humans and Animals. Alan Yuille (UCLA) Detecting and Parsing of Visual Objects: Humans and Animals Alan Yuille (UCLA) Summary This talk describes recent work on detection and parsing visual objects. The methods represent objects in terms of

More information

Unsupervised Co-Segmentation of 3D Shapes via Affinity Aggregation Spectral Clustering

Unsupervised Co-Segmentation of 3D Shapes via Affinity Aggregation Spectral Clustering Unsupervised Co-Segmentation of 3D Shapes via Affinity Aggregation Spectral Clustering Zizhao Wu a, Yunhai Wang b, Ruyang Shou a, Baoquan Chen b, Xinguo Liu a Corresponding author: cloudseawang@gmail.com

More information

Data-Driven Structural Priors for Shape Completion

Data-Driven Structural Priors for Shape Completion Data-Driven Structural riors for Shape Completion Minhyuk Sung 1 Vladimir G. Kim 1,2 Roland Angst 1,3 Leonidas Guibas 1 1 Stanford University 2 Adobe Research 3 Max lanck Institute for Informatics a1.

More information

CS468: 3D Deep Learning on Point Cloud Data. class label part label. Hao Su. image. May 10, 2017

CS468: 3D Deep Learning on Point Cloud Data. class label part label. Hao Su. image. May 10, 2017 CS468: 3D Deep Learning on Point Cloud Data class label part label Hao Su image. May 10, 2017 Agenda Point cloud generation Point cloud analysis CVPR 17, Point Set Generation Pipeline render CVPR 17, Point

More information

DEPARTMENT OF COMPUTER SCIENCE

DEPARTMENT OF COMPUTER SCIENCE Department of Computer Science 1 DEPARTMENT OF COMPUTER SCIENCE Office in Computer Science Building, Room 279 (970) 491-5792 cs.colostate.edu (http://www.cs.colostate.edu) Professor L. Darrell Whitley,

More information

LEARNING TO INFER AND EXECUTE 3D SHAPE PROGRAMS

LEARNING TO INFER AND EXECUTE 3D SHAPE PROGRAMS LEARNING TO INFER AND EXECUTE 3D SHAPE PROGRAMS Anonymous authors Paper under double-blind review ABSTRACT Human perception of 3D shapes goes beyond reconstructing them as a set of points or a composition

More information