Project Gotham Racing 2 (Xbox) Real-Time Rendering. Microsoft Flighsimulator. Halflife 2

Transcription

1 Project Gotham Racing 2 (Xbox) Real-Time Rendering Microsoft Flighsimulator Halflife 2 1

2 Motivation (1) Many graphics applications are dynamic Simulators (surgery, flight simulators, ) 3D computer games 3D walk through s Minimal frame rates 10 Hz: perceived as animation 60 Hz: flicker free 2 2

3 Motivation (2) Geometry (vertex count) 200 Million 83 Million 0.14 Million Elevation model of Switzerland (Endoxon AG) Double Eagle Tanker (UNC Walkthru Group) Brooks House (UNC Walkthru Group) Display resolution (pixel count) e.g x 1000 = 2 Million 3 3

4 Motivation (3) Today s best peak performance 600 M vertices/s 10 M vertices/frame! 6 G fragments/s 100 M fragments/frame! Reality Latest hardware not available on all systems Slowdown by complex shader programs Multi pass rendering Non perfect load balancing Speedup shading & reduce geometry 4 4

5 Overview Shading Acceleration Optimization of Graphics Pipeline Geometry Reduction Spatial Data Structures Culling Techniques Levels of Detail 5 5

6 Simplified Graphics Pipeline CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer Texture Storage + Filtering Vertices Pixels Pipeline slides courtesy John Spitzer - NVIDIA 6 6

7 Possible Pipeline Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 7 7

8 Locate the bottleneck(s) Battle Plan for Better Performance Eliminate the bottleneck (if possible) Decrease workload of the bottlenecked stage Otherwise, balance the pipeline Increase workload of the non-bottlenecked stages 8 8

9 Run App Bottleneck Identification Vary Frame Buffer FPS varies? No Yes Frame Buffer limited Vary texture size/filtering Vary resolution Vary vertex instructions FPS varies? No FPS varies? No FPS varies? No Yes Yes Yes Texture limited Vary fragment instructions Transform limited Yes FPS varies? No Fragment limited Raster limited Vary vertex size/ AGP rate FPS varies? No Yes Transfer limited CPU limited 9 9

10 CPU Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 10 10

11 CPU Bottlenecks Application limited simulation, disc I/O slower that rendering tune application (profiling tools) Driver or API limited too many state changes too many API calls use arrays, display lists 11 11

12 Geometry Transfer Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 12 12

13 Vertex data problems size issues (just under or over 32 bytes) non-native types (e.g. double, packed byte normals) Using the wrong API calls immediate mode, non-accelerated vertex arrays use vertex arrays + indexed primitives (shared vertices transferred only once) access vertices in sequential (cache friendly) pattern AGP misconfigured (rare) Geometry Transfer Bottlenecks 13 13

14 Geometry Transform Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 14 14

15 Too many vertices Geometry Transform Bottlenecks Use levels of detail (but beware of CPU overhead) Use bump maps to fake geometric detail Too much computation per vertex Move per-object calculations to CPU, save results as constants Make use of full spectrum of instruction set (LIT, DST, SIN,...) Make use of swizzle and mask operators to minimize MOVs 15 15

16 Rasterization Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 16 16

17 Rasterization Bottlenecks Rarely the bottleneck Speed influenced by number of vertex attributes to be interpolated Be sure to maximize depth culling efficiency Clear depth at the beginning of each frame Coarsely sort objects front to back Constrain near and far planes to geometry visible in frame 17 17

18 Texture Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 18 18

19 Texture Bottlenecks Running out of texture memory Texture resolutions only as big as needed Avoid expensive internal formats (floating point) Compress textures (16-bit color depth for environment maps and shadow maps) Poor texture cache utilization Localize texture accesses Use expensive filtering (trilinear, anisotropic) only when needed 19 19

20 Fragment Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 20 20

21 Fragment Bottlenecks Too many fragments Consider using a depth-only first pass shade only the visible fragments in subsequent pass(es) Too much computation per fragment Use a mix of texture and math instructions (they often run in parallel) Move constant per-triangle calculations to vertex program, send data as texture coordinates Do similar with values that can be linear interpolated 21 21

22 Framebuffer Bottlenecks CPU transfer transform raster texture fragment frame buffer CPU Geometry Storage Geometry Processor Rasterizer Fragment Processor Frame Buffer CPU/Bus Bound Texture Storage + Filtering Vertex Bound Pixel Bound 22 22

23 Minimizing Framebuffer Traffic Reduce number of passes with longer shaders Floating point frame buffers only when needed Use 16-bit Z-depth if you can get away with it 23 23

24 Overview Shading Acceleration Optimization of Graphics Pipeline Geometry Reduction Spatial Data Structures Culling Techniques Levels of Detail 24 24

25 Spatial Data Structures Key to geometry reduction: Bounding Volume Hierarchies Space Partitioning Trees Scene Graphs 25 25

26 Bounding Volumes Bounding Volume (BV) Volume that encloses a set of objects Should be much simpler geometrical shape than contained objects Sphere ABB OBB 6-DOP Convex hull simplicity quality 26 26

27 Bounding Volume Hierarchies Tree structure Leaf nodes contain geometry + bounding volume Internal nodes contain bounding volume of subtree root Internal node 27 27

28 Space Partitioning Trees Uniform subdivision Quadtree k-d tree BSP tree 28 28

29 Creation Binary Space Partitioning (BSP) Trees Use plane to divide the space in two subspaces Sort geometry into these two subspaces Divide subspaces recursively Types Plane axis or polygon aligned Used in many game engines: Quake I/II/III, Doom I/II, Unreal, Sin, Half-Life etc

30 Axis-Aligned BSP Trees B D E 0 1a 1b plane 1a plane 1b A C 30 plane 2 plane 0 A B C 2 k-d trees: cycle through axes Applications: Rough front-to-back sorting Occlusion culling D E 30

31 Polygon-Aligned BSP Trees A polygon is chosen as the divider V F C A A B D E B C D E F G 31 splitting plane G Creation time consuming Applications: Special type in painter s algorithm (no Z-buffer needed) Intersection testing Collision detection 31

32 Octrees Box is split simultaneously along all three axes Split point must be center of box Creates eight new boxes, hence the name In 2D: Quadtrees Regular structure, thus efficient Applications: View frustum culling Occlusion culling Efficient ray intersections 32 32

33 Scene Graphs Higher level data structure Geometry, augmented with textures, transforms, levels-of-detail, light sources Tree Representation Depth-first traversal for rendering Root node often has a BV Leaf stores geometry Several nodes may point to same child node, thus the tree structure is a Directed Acyclic Graph (DAG) 33 33

34 Overview Shading Acceleration Optimization of Graphics Pipeline Geometry Reduction Spatial Data Structures Culling Techniques Levels of Detail 34 34

35 Culling Techniques Don t render what can t be seen Off-screen: view-frustum culling Occluded by other objects: occlusion culling Why in software? Off-screen geometry: solved by clipping Occluded geometry: solved by Z-buffer 35 35

36 Culling Techniques Save rendering time Clipping and Z-buffering take time linear to the number of primitives Culling in software: Pre-culling of large number of primitives Approximative Conservative! Keep hidden primitive OK, remove visible primitive not OK Remainder is potentially visible set or PVS Do clipping and Z-buffering only with PVS 36 36

37 Visibility From a Point Terminology viewpoint occluder shadow volume = umbra occludee 37 37

38 Occluder Fusion Occluder Fusion combines effects of multiple occluders viewpoint larger shadow volume 2 occluders Visible without occluder fusion Invisible with occluder fusion 38 38

39 viewing region Visibility From a Region 1 occluder shadow volume XXX 2 occluders 39 39

40 viewing region Visibility From a Region (2) shadow volume 2 occluders shadow volume XXX 40 40

41 Visibility From a Region (3) viewing region 2 occluders shadow volume XXX Area XXX is always occluded -> complete shadow volume is more than the union of individual shadow volumes 41 41

42 Bounding Planes and Separating Planes bounding planes viewing region occluder separating planes Separating planes of two convex polyhedra are planes formed by an edge of one polyhedron and a vertex of the other such that the polyhedra lie on opposite sides of the plane Bounding planes are similar, except that both polyhedra lie on the same side of the plane 42 42

43 Umbra and Penumbra penumbra umbra viewing region occluder penumbra The area (volume) in full shadow is the umbra Simple in/out classification The area not in full shadow (grey) is the penumbra Additionally encodes which part of the viewing region is visible 43 43

44 Occlusion Culling From a Region penumbra umbra viewing region occluder penumbra We want to calculate the complete umbra = all regions that are fully occluded Umbra = umbra for each individual occluder + all regions where penumbra merges to umbra 44 44

45 Example Culling Methods Backface culling View frustum culling Portal culling Occlusion culling 45 45

46 Backface Culling On the surface of a closed manifold, polygons whose normals point away from the camera are always occluded Viewer Note: backface culling alone doesn t solve the hidden-surface problem! 46 46

47 Clustered Backface Culling Group objects with similar normal Classification of objects: All faces front-facing -> render all All faces back-facing -> render none Some front-facing, some back-facing -> render all Done with normal cones Collected normals from a set of polygons n α Minimal normal cone defined by one normal n and a half-angle α 47 47

48 Hierarchical View Frustum Culling intersect intersect inside inside inside outside inside inside inside Bounding Volume Hierarchy 48 48

49 Portal Culling Goal: walk through architectural models (buildings, cities, catacombs) These divide naturally into cells Rooms, alcoves, corridors Transparent portals connect cells Doorways, entrances, windows 49 49

50 Example viewer Portal Culling (2) 50 50

51 Portal Culling (3) Idea: Cells form the basic unit of PVS Create an adjacency graph of cells Starting with cell containing eyepoint, traverse graph, rendering visible cells A cell is only visible if it can be seen through a sequence of portals So cell visibility reduces to testing portal sequences for a line of sight 51 51

52 Occlusion Culling Three main algorithms: Loose front-to-back sorting Using an octree or BSP tree Sorting centroids or near points of bounding volumes Hierarchical Z-buffer Ned Greene, SIGGRAPH 93 Hierarchical occlusion maps Hansong Zhang, SIGGRAPH 97, not discussed 52 52

53 Hierarchical Z-buffering Replace Z-buffer with a Z-pyramid Lowest level: full-resolution Z-buffer Higher levels: each pixel represents the maximum depth of the four pixels underneath it Basic idea: hierarchical rasterization of the polygon, with early termination where polygon is occluded 53 53

54 Hierarchical Z-buffering (2) Idea: test polygon against highest level first If polygon is further than distance recorded in pixel, stop it s occluded If polygon is closer, recursively check against next lower level If polygon is visible at lowest level, set new distance value and propagate up Unfortunately: no hardware support 54 54

55 Overview Shading Acceleration Optimization of Graphics Pipeline Geometry Reduction Spatial Data Structures Culling Techniques Levels of Detail 55 55

56 Level of Detail What for? Even after visibility, model may contain too many polygons Idea: reduce complexity further by managing level of detail (LOD) Multi-resolution modeling Polygonal simplification 56 56

57 Level of Detail Traditional LOD Create levels of detail (LODs) of objects 69,451 polys 2,502 polys 251 polys 76 polys Courtesy Stanford 3D Scanning Repository 57 57

58 Level of Detail Traditional LOD Distant objects use coarser LODs Changes number of vertices Not number of fragments 58 58

59 Level of Detail Issues LOD generation Lots of methods LOD selection Static (range/projected area-based) Reactive (react to last frames rendering time) Predictive (cost/benefit model) LOD switching Hard switching (popping artifacts!) Blending (ill-defined because of Z-buffer!) Geomorph (for continuous level of detail) 59 59

60 Level of Detail LOD Generation Lots of methods Decimation Vertex Merging (edge-collapsing) Adaptive subdivision Interesting: Different error metrics! 60 60

61 Level of Detail LOD Selection Range-based Associate different LODs with different ranges LOD 0 LOD 1 LOD 2 LOD 3 r 1 r 2 r

62 Level of Detail LOD Selection Projected area-based Projected area of the bounding volume Estimation of screen space coverage E.g. spheres: Area of projection is thus p = nr d ( c v) 2 πp Higher value selects a more detailed LOD v d n c r 62 62

63 Level of Detail LOD Switching Discrete Geometry LODs Create LODs for each object separately in a preprocess At run-time, pick each object s LOD according to the object s distance (or similar criterion) Since LODs are created offline at fixed resolutions, we call this discrete LOD Decouples simplification and rendering 63 63

64 Level of Detail LOD Switching (2) Continuous LODs Create data structure from which a desired level of detail can be extracted at run time. Better granularity better fidelity LOD is specified exactly, not chosen from a few pre-created options Thus objects use no more polygons than necessary, which frees up polygons for other objects Net result: better resource utilization, leading to better overall fidelity/polygon 64 64

65 Level of Detail LOD Switching (3) Continuous LODs Better granularity -> smoother transitions Switching between traditional LODs can introduce visual popping effect Continuous LOD can adjust detail gradually and incrementally, reducing visual pops Can even geomorph the fine-grained simplification operations over several frames to eliminate pops [Hoppe 96, 98] 65 65

66 Level of Detail LOD Switching (4) Continuous LODs (2) Supports progressive transmission Progressive Meshes [Hoppe 97] Progressive Forest Split Compression [Taubin 98] Leads to view-dependent LOD Use current view parameters to select best representation for the current view Single objects may thus span several levels of detail 66 66

67 Level of Detail Conclusions Good book and webpage: lodbook.com 10k 20k 60k 200k 2000k Michelangelo s David at different levels of detail [Pauly 02] In points of the original model, rendered with a point splatter

68 Acceleration Algorithms Conclusion Most visibility algorithms work on the data before it gets down the graphics pipeline Ideal optimizations Problematic if CPU is bottleneck Most of the work in LODs can be done in a preprocessing phase 68 68