(Urban) Forward Procedural and Inverse Procedural Modeling

Transcription

1 (Urban) Forward Procedural and Inverse Procedural Modeling Daniel Aliaga, CGVLAB Department of Computer Science Purdue University

2 What is the main challenge? Solving the content problem As computing and display capabilities continually improve, audience expects ever higher quality digital content Traditional tools are insufficient for increasing demand and few tools are available for efficient large-scale modeling

3 What is procedural modeling? One content generation option is modeling buildings, plants, clouds, cities, and worlds using an underlying system of rules. G = { terminals, non-terminals rules, axiom }

4 Procedural Modeling Fundamentals Example Works Outline Inverse Procedural Modeling Fundamentals Pipeline and Example Works

5 Grammars A rule-based generative system loosely organized as:

6 L-systems e.g., plants Split Grammars e.g., facades, buildings Shape Grammars Grammars e.g., architecture, sculptures, terrain, CGA (CityEngine)

7 L-systems A parallel string rewriting system Generation of plants Prusinkiewicz, Lindenmayer; 1990 Environment-sensitive Prusinkiewicz, James, Mech; 1994 Interaction (Open L-System) Mech, Prusinkiewicz; 1996 Ecosystems Deussen, et al.; 1998

8 L-system

9 L-systems

10 L-systems e.g., plants Split Grammars e.g., facades, buildings Shape Grammars Grammars e.g., architecture, sculptures, terrain, CGA (CityEngine)

11 Split Grammars Recursively split plane/volume into subparts

12 L-systems e.g., plants Split Grammars Some Grammars e.g., facades, buildings Shape Grammars e.g., architecture, sculptures, terrain, CGA (CityEngine)

13 Shape Grammar Rules for how a shape can be transformed

14 Shape Grammar Rules for how a shape can be transformed

15 Fractals Other Generative Models e.g., diamond-square algorithm Procedural Noise e.g., Perlin Noise And more! e.g., continuous shape spaces, learned models

16 Procedural Modeling: Example Seek a grammar that produces a desired 3D model

17 Procedural Modeling: Example Imagine the output Create terminals and rules Write grammar R 4 T 5 R 3 T 6 R 1 R 2 Rule Building(S) = { // CREATE ENTIRE BUILDING a = split(s, 0.0, 0.0, 0.80, 4) // roof terminal c = split(s, 1.0, 1.0, 0.01, 5) // floor terminal B = split(s, 1.0, 1.0, 0.80, 5) // wall bbox D = split(b, 1.0, 0.1, 1.00, 5) // front wall bbox E = split(b, 0.0, 0.9, 0.00, 4) // back wall bbox F = split(b, 0.1, 1.0, 1.00, 5) // left wall bbox G = split(b, 0.9, 0.0, 0.00, 4) // right wall bbox R3(D); R3(E); // front/back rule R4(F); R4(G); } // left/right rule } Rule R3(A) = { // CREATE FRONT/BACK WALL b = split(a, 0.0, 0.0, 0.00, 4) // wall terminal C = split(a, 0.4, 1.0, 0.75, 5) D = split(c, 0.6, 0.3, 0.00, 4) // left window bbox E = split(a, 0.6, 1.0, 0.75, 6) F = split(e, 0.3, 1.0, 0.30, 1) // right window R1(D); R1(F); // window rule } Rule R1(A) = { // perform splits for frame and window } [...and add l text for R2 and R4] Derive an instance based on desired model parameters

18 Procedural Modeling: Example Ultimately, to produce this building need to write Rule Building(S) = { // CREATE ENTIRE BUILDING a = split(s, 0.0, 0.0, 0.80, 4) // roof terminal c = split(s, 1.0, 1.0, 0.01, 5) // floor terminal B = split(s, 1.0, 1.0, 0.80, 5) // wall bbox D = split(b, 1.0, 0.1, 1.00, 5) // front wall bbox E = split(b, 0.0, 0.9, 0.00, 4) // back wall bbox F = split(b, 0.1, 1.0, 1.00, 5) // left wall bbox G = split(b, 0.9, 0.0, 0.00, 4) // right wall bbox R3(D); R3(E); // front/back rule R4(F); R4(G); } // left/right rule } Rule R3(A) = { // CREATE FRONT/BACK WALL b = split(a, 0.0, 0.0, 0.00, 4) // wall terminal C = split(a, 0.4, 1.0, 0.75, 5) D = split(c, 0.6, 0.3, 0.00, 4) // left window bbox E = split(a, 0.6, 1.0, 0.75, 6) F = split(e, 0.3, 1.0, 0.30, 1) // right window R1(D); R1(F); // window rule } Rule R1(A) = { // perform splits for frame and window } [...and add l text for R2 and R4]

19 Procedural Modeling: CityEngine #lines output simplebuilding.04.cga 171 lines CandlerBuilding.cga 471 lines parthenon.cga 524 lines

20 Procedural Modeling: CityEngine version "2010.3" /* Attributes attr groundfloor_height = attr floor_height = attr tile_width = attr height = rand(11,30) attr wallcolor = 33% : "#ffffff" 33% : "#999999" else : attr LOD = 0 /* Assets *************************************/ // geometries const window_asset = "facades/window.obj" // textures const frontdoor_tex = "facades/textures/shopdoor.tif" const wall_tex = "facades/textures/brickwall.jpg" const dirt_tex = "facades/textures/dirtmap.15.tif" const roof_tex = "roofs/roof.tif" # this function will get one of the 9 window textures in the assets folder randomwindowtexture = filerandom("*facades/textures/window.*.tif") /* Initial Shape starting rule *************/ # scale the lit to leave a small border and extrude the lot to building height Lot --> extrude(height) Building # inner lots are dropped LotInner --> NIL # split the building geometry into its facade components Building --> comp(f) { front : Frontfacade side : Sidefacade top: Roof} # the front facade is subdivided into one front groundfloor # and upper floors Frontfacade --> setupprojection(0, scope.xy, 1.5, 1, 1) # setup 1.5m x 1m texture tiles along scopes xy plane (and distortion in z) setupprojection(2, scope.xy, scope.sx, scope.sy) split(y){ groundfloor_height : Groundfloor {~floor_height : Floor}* } # a side facade is subdivided into one bottom floor # and upper floors. Sidefacade --> setupprojection(0, scope.xy, 1.5, 1, 1) # setup 1.5m x 1m texture tiles along scopes xy plane (and distortion in z) setupprojection(2, scope.xy, scope.sx, scope.sy) split(y){ groundfloor_height : Floor {~floor_height : Floor}* } # a roof texture is applied to the roof face Roof --> color(wallcolor) setupprojection(0, scope.xy, scope.sx, scope.sy) texture(roof_tex) projectuv(0) # each floor is horizontally split into two narrow corner areas on # each side of the floor, and into a set of window tiles in between Floor --> split(x){ 1 : Wall { ~tile_width : Tile }* 1 : Wall } attr BuildingHeight @Description("Generates a watertight model with less detail - suited for most 3D printers.") attr Mode @Range(2.5,5) attr FloorHeight attr GroundfloorHeight attr TileWidth attr CorniceOverhang @Range(1.3,5) attr attr attr attr @Range(0,2) attr attr WallTexture= attr ColorizeWall= attr onstreet= true const winsidew= (TileWidth-FrontWindowWidth)/2 const dividerw= 0.25 const sillh= case FloorHeight>WindowHeight+SillSize: (FloorHeight-WindowHeight)*0.66 else: SillSize # # Extrusion and facade dispatcher # Footprint --> InsideSkin extrude(buildingheight) Solid Solid --> comp(f){ front: FrontFacade back : RearFacade left : SideFacade right: SideFacade top : Roof } FrontFacade --> set(onstreet,true) Facade(2,1,3,1,2) RearFacade --> set(onstreet,false) # is not facing street (i.e. no shops nor cornice) Facade(0,0, 0,0,0) # small windows only (marked with 0) SideFacade --> case scope.sx < 30: # catching inner backfacades (hack due to the uncommon footprint) RearFacade else: set(onstreet,true) # is facing street (i.e. has shops, ledges and cornice) Facade(2,1,3,2,3)

21 Procedural Modeling: Observations Given a target object/shape, the PM system must be hand-crafted PM provides high detail amplification Pro: lots of content Con: its hard to write a PM system

22 Procedural Modeling Fundamentals Example Works Outline Inverse Procedural Modeling Fundamentals Pipeline and Example Works

23 What is inverse procedural modeling? Extracting rules and/or parameters to create a generative system from given buildings, plants, clouds, cities, and worlds. G = { terminals, non-terminals rules, axiom }

24 What are the current approaches? Having low-level priors: Formulations to partition geometry into components and then extract patterns and rules

25 What are the current approaches? Having low-level priors: Formulations to componentize the geometry into similarity groups to extract the patterns from. Having high-level priors: Guiding and controlling the content generation by discovering the optimal parameters and rules for a given target.

26 Inverse Procedural Modeling (IPM) Converts existing geometry into procedural models: IPM PM Geometry Procedural Model (Grammar) Content Parameters

27 IPM Classification Converts existing geometry into procedural models A classification: 1. inferring procedural model and its parameters Geometry? (Grammar) Content Parameters?

28 IPM Classification Converts existing geometry into procedural models A classification: 2. inferring procedural model parameters (i.e., rules are known) Geometry Procedural Model (Grammar) Content Parameters?

29 IPM Classification Inferring procedural model and its parameters - Only low-level priors can be assumed - Usually operates directly on the geometry Inferring procedural model parameters - High-level priors can be assumed - PM system is given but its usage must be determined/searched

30 Benefits of IPM Content Synthesis - Provides a parameterized procedural representation for easy synthesis of new content Reconstruction - Can fill-in the details Rendering and Compaction - Can compress the model and lead to faster rendering times and smaller model footprint

31 Procedural Modeling Fundamentals Example Works Outline Inverse Procedural Modeling Fundamentals Example Works

32 Procedural Modeling Pipeline Input Major Roads Roads Minor Roads Blocks Lots Mass Buildings Aerial Views Facades Input Target architectural design Example 3D model/gis data/imagery Socioeconomic data/elevation data Tensor field

33 Procedural Modeling Pipeline Input Major Roads Roads Minor Roads Blocks Lots Mass Buildings Aerial Views Facades Road generation Extended L-systems Hyperstreamlines Directed random walks Grow seeds/traffic Simulation

34 Procedural Modeling Pipeline Input Major Roads Roads Minor Roads Blocks Lots Mass Buildings Aerial Views Facades Blocks and Lots Recursive block subdivision Voronoi-diagram based subdivision

35 Procedural Modeling Pipeline Input Major Roads Roads Minor Roads Blocks Lots Mass Buildings Aerial Views Facades Buildings Shape/split grammars Mass/façade modeling Build by numbers Image-based synthesis

36 Cities Procedural Modeling of Cities - Parish and Müller SIGGRAPH 2001

37 Cities Continuous Model Synthesis - Merrell, Manocha SIGGRAPH Asia 2008

38 Cities On-the-fly Generation and Rendering of Infinite Cities on the GPU - Steinberger, Kenzel, Kainz, Wonka, Schmalstieg Eurographics 2014

39 Layouts Example-based Urban Layout Synthesis - Aliaga, Vanegas, Benes SIGGRAPH Asia 2008

40 Layouts Urban Pattern: Layout Design by Hierarchical Domain Splitting Yang, Wang, Vouga, Wonka SIGGRAPH Asia 2013

41 Layouts Computational Network Design from Functional Specifications - Peng, Yang, Bao, Fink, Yan, Wonka, Mitra SIGGRAPH 2016

42 Buildings Procedural Modeling of Buildings - Müller, Wonka, Haegler, Ulmer, Van Gool SIGGRAPH 2006

43 Buildings Interactive Visual Editing of Grammars for Procedural Architecture - Lipp, Wonka, Wimmer SIGGRAPH 2008

44 Buildings Interactive Architectural Modeling with Procedural Extrusions - Kelly, Wonka SIGGRAPH 2011

45 Buildings Interactive Design of Probability Density Functions for Shape Grammars - Dang, Lienhard, Ceylan, Neubert, Wonka, Pauly SIGGRAPH Asia 2015

46 Buildings Advanced Procedural Modeling of Architecture - Schwarz, Mueller SIGGRAPH 2015

47 Buildings Learning Shape Placements by Example - Guerrero, Jeschke, Wimmer, Wonka SIGGRAPH 2015

48 Buildings Design Transformations for Rule-based Procedural Modeling - Lienhard, Lau, Mueller, Wonka, Pauly Eurographics 2017

49 Facades Instant Architecture - Wonka, Wimmer, Sillion, Ribarsky SIGGRAPH 2003

50 Facades Layer-Based Procedural Design of Facades - Ilcik, Musialski, Auzinger, Wimmer Eurographics 2015

51 Parcels Procedural Generation of Parcels in Urban Modeling - Vanegas, Kelly, Weber, Halatsch, Aliaga, Mueller Eurographics 2012

52 Terrains WorldBrush: Interactive Example-based Synthesis of Procedural Virtual Worlds - Emilien, Vimont, Cani, Poulin, Benes SIGGRAPH 2015

53 Trees ShapeGenetics: Using Genetic Algorithms for Procedural Modeling - Haubenwallner, Seidel, Steinberger Eurographics 2017

54 Procedural Modeling Fundamentals Example Works Outline Inverse Procedural Modeling Fundamentals Example Works

55 Inverse Modeling Pipeline

56 Inverse Modeling Pipeline

57 Inverse Modeling Pipeline

58 Inverse Modeling Pipeline

59 Inverse Modeling Pipeline A C S B

60 A B C B C B C B A F = ABCBCBCBA Inverse Modeling Pipeline F A(BC)*BA

61 Cities Metropolis Procedural Modeling - Talton, Lou, Lesser, Duke, Mech, Koltun SIGGRAPH 2011

62 Cities Inverse Design Of Urban Procedural Models - Vanegas, Aliaga, Benes SIGGRAPH 2012

63 Cities Proceduralization of Buildings at City Scale - Demir, Aliaga, Benes 3DV 2014 Input model Synthesized city

64 Buildings Style Grammars for Interactive Visualization of Architecture - Aliaga, Rosen, Bekins TVCG 2007

65 Buildings Building Reconstruction using Manhattan- World Grammars - Vanegas, Aliaga, Benes CVPR 2010

66 Buildings A Connection Between Partial Symmetry and Inverse Procedural Modeling - Bokeloh, Wand, Seidel SIGGRAPH 2010

67 Buildings Procedural Editing of 3D Building Point Clouds - Demir, Aliaga, Benes ICCV 2015

68 Buildings Procedural 3D Building Reconstruction Using Shape Grammars And Detectors - Mathias, Martinovic, Weissenberg, van Gool 3DIMPVT 2011

69 Buildings Interactive Sketching of Urban Procedural Models - Nishida, Garcia-Dorado, Aliaga, Benes, Bousseau SIGGRAPH 2016

70 Facades Image-based Procedural Modeling of Facades - Müller, Zeng, Wonka, Van Gool SIGGRAPH 2007

71 Facades Cityfit: High-quality Urban Reconstructions By Fitting Shape Grammars To Images And Derived Textured Point Clouds - Hohmann, Krispel, Havemann, Fellner ISPRS 2009

72 Facades Inverse Procedural Modeling of Facade Layouts - Wu, Yan, Dong, Zhang, Wonka SIGGRAPH 2013

73 Trees Inverse Procedural Modeling of Trees Stava, Pirk, Kratt, Chen, Mech, Deussen, Benes Eurographics 2014

74 Urban Traffic Designing Large-Scale Interactive Traffic Animations for Urban Modeling Garcia-Dorado, Aliaga, Ukkusuri Eurographics 2014

75 Urban Weather Inverse Procedural Weather Design for 3D Urban Models Garcia-Dorado, Aliaga, Bhalachandran, Schmid, Niyogi TOG 2017

76 Conclusions On the one hand Procedural modeling is powerful but requires writing detailed grammars On the other hand There are many model databases (and capture strategies) which lack structural and functional representations

77 Conclusions Thus inverse procedural modeling is also an exciting and growing area of research!

78 Conclusions A Multidisciplinary Goal: Take advantage of multiple strategies to enable modelers to automatically generate new content in various flexible ways

79 Thank you! Questions?