Finding Structure in Large Collections of 3D Models

Transcription

1 Finding Structure in Large Collections of 3D Models Vladimir Kim Adobe Research

2 Motivation Explore, Analyze, and Create Geometric Data Real Virtual

3 Motivation Explore, Analyze, and Create Geometric Data Personal data (image from naked.fit)

4 Motivation Explore, Analyze, and Create Geometric Data Scans of environments Personal data (image from naked.fit)

5 Scientific data Motivation Explore, Analyze, and Create Geometric Data Scans of environments Personal data (image from naked.fit) Medical Imaging

6 Motivation Explore, Analyze, and Create Geometric Data CG data (image from 3dwarehouse.sketchup.com)

7 Motivation Explore, Analyze, and Create Geometric Data CG data (image from 3dwarehouse.sketchup.com) 3D printing data (image from thingiverse.com/)

8 Motivation Explore, Analyze, and Create Geometric Data CG data (image from 3dwarehouse.sketchup.com) 3D printing data (image from thingiverse.com/) CAD models

9 Goal Structural Model Applications 3D Data

10 Goal Structural Model Explore 3D Data Organize and explore large collections of shapes

11 Goal Structural Model Analyze 3D Data Understand and label novel geometric data

12 Goal Structural Model Create 3D Data Make 3D modeling more accessible to non-experts

13 Goal Structural Model Create Discover repeating patterns and represent them 3D Data with a concise model Make 3D modeling more accessible to non-experts

14 Diversity Scale Ambiguity Challenges Discover similarities in unorganized geometry Point-to-point Region-to-region Part-to-part

15 Diversity Scale Ambiguity Challenges Discover similarities in unorganized geometry Point-to-point Region-to-region Part-to-part

16 Diversity Scale Ambiguity Challenges Discover similarities in unorganized geometry Point-to-point Region-to-region Part-to-part

17 Talk Outline Discover similarities to model structure Discover similarities - Blended intrinsic maps - Soft maps Model the structure - Learning deformable templates - Learning hierarchical templates

18 Talk Outline Discover similarities to model structure Discover similarities - Blended intrinsic maps - Soft maps Model the structure - Learning deformable templates - Learning hierarchical templates

19 Shape Correspondence Correspondence as Metric Alignment : M 1! M 2

20 Shape Correspondence Correspondence as Metric Alignment q p (q) (p) a good map: arg min kd(p, q) D( (p), (q)k

21 Shape Correspondence Correspondence as Metric Alignment Sample >> Pick best q p (q) (p) a good map: arg min kd(p, q) D( (p), (q)k

22 Shape Correspondence Correspondence as Metric Alignment Sample >> Pick best q Observation: p (q) isometric = area-preserving + conformal (p) a good map: arg min kd(p, q) D( (p), (q)k

23 Shape Correspondence Correspondence as Metric Alignment q Sample Observation: >> Pick best combinatorial! p (q) isometric = area-preserving + conformal (p) a good map: arg min kd(p, q) D( (p), (q)k

24 Shape Correspondence Correspondence as Metric Alignment Sample >> Pick best q Observation: p (q) (p) isometric = area-preserving + conformal polynomial! a good map: arg min kd(p, q) D( (p), (q)k

25 Conformal Maps Correspondence as Metric Alignment Sample >> Pick best q p (q) (p) conformal map defined by 3 points a good map: arg min kd(p, q) D( (p), (q)k

26 Conformal Maps Correspondence as Metric Alignment Sample >> Pick best q distortion p (q) (p) conformal map defined by 3 points a good map: arg min kd(p, q) D( (p), (q)k

27 Blended Conformal Maps Conformal Maps Weights Blended Map

28 Blended Conformal Maps Non-isometric shapes

29 Limitations What is the correct point-to-point map?

30 Limitations 3D model 2D outline What is the correct point-to-point map?

31 Soft Maps

32 Soft Maps Correspondence as Metric Alignment Sample >> Pick best q a b c p (q) (p) q p mass transport point-to-distribution a good map:

33 Soft Maps Correspondence as Metric Alignment Sample >> Pick best q a b c p (q) (p) q p mass transport point-to-distribution a good map: arg min kd(p, q) D(a, b)k 2 (p, a) (q, b)

34 Entropic Regularization Make as sparse as possible: H ( )= <, ln >

35 Soft Map Results

36 Soft Map Results

37 Consistency Low-rank (cycle consistency) in collections of shapes

38 Consistency Low-rank assumption can help in discovering parts Additional terms: local shape cues for part boundaries

39 Talk Outline Discover similarities to model structure Discover similarities - Blended intrinsic maps - Soft maps Model the structure - Learning deformable templates - Learning hierarchical templates

40 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates

41 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates Deformable Template

42 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates Deformable Template

43 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates Deformable Template

44 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates Deformable Template

45 Learning Templates Meta-algorithm Shape-shape corrs. Build templates Shape-template corrs. Update templates Deformable Template

46 Hierarchical Templates Use hierarchical templates for scenes

47 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling

48 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling

49 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling

50 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling Input

51 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling Input Estimated Structure

52 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling Input Estimated Structure Output

53 Applications Input Output

54 Applications Input Output

55 Applications Explore, Analyze, and Create Geometric Data Region-based exploration Scene completion 3D Modeling

56 Summary Large collections of 3D models are available (and more are coming!) o 3D modeling repositories o 3D printing datasets o Kinect scans o Laser scans of cities, o Online shopping catalogues o Medical imaging data o Protein databases o

57 Summary Large collections of 3D models are available (and more are coming!) Discovering similarities is essential to find structure o Need to handle: diversity, scale, and ambiguity in data

58 Summary Large collections of 3D models are available (and more are coming!) Discovering similarities is essential to find structure Finding structure can enable novel techniques for: o Exploring and organizing the data o Scene understanding o Modeling

59 Future Work Beyond geometry Understand function Obtain human supervision to capture what is semantically salient Consider physical materials that make up objects Model mechanics and articulations

60 Future Work Beyond geometry Understand function Obtain human supervision to capture what is semantically salient Consider physical materials that make up objects Model mechanics and articulations Predict human-object interaction pose for shape analysis

61 Collaborators R. Angst M. Averkiou Q. Huang S. Chaudhuri S. DiVerdi T. Funkhouser L. Guibas W. Li Y. Lipman N. Mitra G. Peyre J. Solomon S. Sra M. Sung Y. Zheng

62 References Blended Intrinsic Maps, V. Kim et al., SIGGRAPH 2011 Entropic Metric Alignment for Correspondence Problems J. Solomon et al. SIGGRAPH 2016 Exploring Collections of 3D Models using Fuzzy Correspondences V. Kim et al., SIGGRAPH 2012 Learning Part-based Templates from Large Collections of 3D Shapes V. Kim et al., SIGGRAPH 2013 ShapeSynth: Parameterizing Model Collections for Coupled Shape Exploration and Synthesis M. Averkiou et al. Eurographics 2014 Creating Consistent Scene Graphs Using a Probabilistic Grammar T. Liu et al., SIGGRAPH Asia 2014 Shape2Pose: Human-Centric Shape Analysis V. Kim et al., SIGGRAPH 2014 Data-Driven Structural Priors for Shape Completion M. Sung et al. SIGGRAPH Asia 2015