Graphics The Rasterization Pipeline Projection, Visibility, & Shading

Transcription

1 Graphics 2014 The Rasterization Pipeline Projection, Visibility, & Shading

2 Practicals this week Tuesday (today) Announcements Tue 9-11 (was held) Tue canceled (programming contest) Wednesday (tomorrow) Wed 15-17: additional practical slot We 17-19: additional practical slot Thursday: no practicals

3 Addendum: Matrix Algebra CORE core topics important

4 Transposition T = Matrix Transposition Swap rows and columns In other words: Flip around diagonal Formally: a i,j T = a j,i

5 Othogonal Matrices Orthogonal Matrices (i.e., column vectors orthonormal) M T = M 1 Proof: next three slides

6 Matrix Multiplication General matrix products: k B A: possible if #Row(A) = #Columns(B) B m A m n n k R = B A A = B = R = a 1,1 a 1,n a m,1 a m,n b 1,1 b 1,m b k,1 b k,m r 1,1 r 1,n r k,1 r k,n r i,j = m q=1 a q,j b i,q

7 Matrix Multiplication Matrix Multiplication = a 1 a d = A B a i, b j b 1 b d Scalar products of rows and columns

8 Othogonal matrices: = = Matrix Multiplication a 1 a d a i, a j A T A = a 1 a d a 1 = I a 2

9 Transposition Rules Transposition Multiplication: Inversion: A B T = B T A T A B 1 = B 1 A 1 Inverse-transp.: A T 1 = A 1 T Othogonality: A T = A 1 A is orthogonal

10 Homogeneous Coordinates (short version) CORE core topics important

11 Problem Translations are not linear x Mx cannot encode translations Proof: Origin cannot be moved: m 11 m 12 m 13 0 M 0 = m 21 m 22 m 23 0 m 31 m 32 m 33 0 = 0 0 0

12 Homogeneous Coordinates Solution: Just add a constant one Increase dimension R d R d+1 Last entry = 1 in vectors Cheap Trick, Evil Hack M x = = m 11 m 12 m 13 t 1 m 21 m 22 m 23 t 2 m 31 m 32 m 33 t M t x 1 = x y z 1 Mx + t 1

13 Homogeneous Coordinates General case M x = m 11 m 12 m 13 m 14 m 21 m 22 m 23 m 24 m 31 m 32 m 33 m 34 m 41 m 42 m 43 m 44 x y z 1 = x y z w w might be different from 1 Convention: Divide by w-coord. before using Result: x /w y /w z /w 1

14 Homogeneous Coordinates General case M x = m 11 m 12 m 13 m 14 m 21 m 22 m 23 m 24 m 31 m 32 m 33 m 34 m 41 m 42 m 43 m 44 x 1 x 2 x 3 1 = y 1 y 2 y 3 y 4 y 1 /y 4 y 2 /y 4 y 3 /y 4 1 Can express divisions by common denominator Rules: y 4 = m 41 x 1 + m 42 x 2 + m 43 x 3 + m 44 x 4 Before using as 3D point, divide by last (4th) entry No normalization required during subsequent transformations (matrix-mult.)

15 Projective Geometry The Full Story? Not just an evil hack Deep & interesting theoretical background More on this later For simplicity We ll treat it as a computational trick for now Focus on the graphics application Remember for now: We can build 4D Translation matrices for 3D+1 points We can divide by a common linear factor

16 Now for 3D Rendering

17 3D Rendering Overview BASIC basic topics study completely

18 3D Computer Graphics Three main aspects Modeling Describe 3D geometry mathematically From machine parts (e.g., CAD) To natural phenomena (e.g., fractals) Animation Set scenes into motion Simple: Camera fly-through Complex: Fluid simulation, human motion Rendering Convert geometry into images Our Focus right now

19 3D Rendering Assumption 3D Model is given Triangle mesh (for simplicity) How do we get it to the screen?

20 Upcoming Topics Agenda Modeling: mesh representation Physics: Perspective projection Rendering: Two main rendering methods Rasterization Perspective projection Rasterization Visibility Shading Programmable shaders / GPUs Raytracing

21 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

22 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

23 Modeling Mesh Representation BASIC basic topics study completely

24 Modeling Shapes Primitives Elementary geometric building blocks Easy to handle directly Complex models Sets of primitives Approximate shapes with primitives yep, triangles! Most-frequently-used Triangles!

25 Simple Triangle List Vertex list Vector3D vertices[n]; (1) p 1 = (x 1, y 1, z 1 ) (2) p 2 = (x 2, y 2, z 2 ) (3) p 3 = (x 3, y 3, z 3 ) (4) p 4 = (x 4, y 4, z 4 ) Triangle list (int[3]) triangles[m]; (1) t 1 = (i 1, j 1, k 1 ) (2) t 2 = (i 2, j 2, k 2 ) (3) t 3 = (i 3, j 3, k 3 ) (m) t m = (i m, j m, k m ) (n) p n = (x n, y n, z n )

26 Modeling a Triangle Triangle p 2 p 1 t 1 t 2 p 3 Parametric Plane Equation (with constraints) 0 Triangles: x λ, μ = p 1 + λt 1 + μt 2 = p 1 + λ p 2 p 1 + μ p 3 p 1 0 λ 1, 0 μ 1, μ + λ 1

27 Attributes How to define a triangle? We need three points in R 3 But we can have more: per-vertex normal per-vertex color texture per-vertex texture coordinates (etc...)

28 Complete Data Structures Multiple Arrays: Vertices, Triangles, Edges v 1 : (posx posy posz), attrib 1,..., attrib n... v N : (posx posy posz), attrib 1,..., attrib n e 1 : (index 1 index 2 ), attrib 1,..., attrib k... e K : (index 1 index 2 ), attrib 1,..., attrib k edges: optional t 1 : (idx 1 idx 2 idx 3 ), attrib 1,..., attrib m... t M : (idx 1 idx 2 idx 3 ), attrib 1,..., attrib m

29 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

30 Physics Ray Optics & Color CORE core topics important

31 Ray Optics Geometric ray model Light travels along rays

32 Ray Optics Geometric ray model Rays have intensity and color

33 Ray Optics reddish gray with a tint of green bluish 700nm 390nm wavelength λ Color spectrum Continuous spectrum Intensity for each wavelength

34 Human Vision greenish reddish low light (monochrome) bluish Color spectrum 700nm 390nm wavelength λ (curves: schematic, not accurate) Two types of receptive cells (color/low-light) Three types of color cells

35 y-coord. Bitmap (Pixel Display) RGB Model Screen: w h discrete pixels Origin: usually upper left Varying color per pixel RGB Model Every pixel can emit red, green, blue light Intensity range: Usually: bytes = dark 255 = maximum brightness h 1 0 x-coord. w 1 h w

36 Human Vision (ideal) monitor: emitted spectra Response curves: human eye 700nm 390nm wavelength λ Create color impressions Basis for three-dimensional color space Wide spacing, narrow bands: purer colors Otherwise: washed out colors (curves: schematic, not accurate)

37 Physics Perspective Projection BASIC basic topics study completely

38 Pinhole Camera Pinhole camera Create image by selecting rays of specific angles Low efficiency (small holes for sharp images)

39 Pinhole Camera Pinhole camera Create image by selecting rays of specific angles Low efficiency (small holes for sharp images)

40 Central Projection Pinhole Camera

41 Pinhole Camera y y f z Central projection x = f x z Proof: Intercept theorem! y = f y z

42 (Actual Camera) Camera with Lens Higher efficiency (bundles many rays) Finite Depth of field We will consider pinhole cameras only.

43 Pinhole Camera y y x = f x z f z y = f y z Undetermined degree of freedom Focal length vs. image size Source of a lot of confusion!

44 Pinhole Camera α f h Parameters h - size of the screen (pixels, cm, ±1.0,...) f focal length (classical photography) Meaningful parameter: α viewing angle

45 Pinhole Camera α f h Relation: tan α 2 = h 2f

46 Pinhole Camera α h h h f f f Invariance tan α 2 = h 2f = h 2f = h 2f Scaling h and f by a common factor: no change

47 Pinhole Camera α f h Typical choices (vertical angles) Normal perspective: α 30 ( 50mm lens: 27 ) Tele photography: Wide angle lens: α 5 20 (275 70mm) α (28 12mm)

48 General Camera y y x = f x z f z y = f y z Our camera so far: Focus point: origin View direction: z-axis General position/orientation?

49 Homogeneous Coordinates x y z w = f f Projection Matrix P x y z 1 x = f x z y = f y z z = z 1 z w = z Write in homogeneous coordinates Third row is arbitrary (for now), not used.

50 View transform α h h f Reminder: tan α 2 = h 2f

51 To Screen Coordinates 1 tan α tan α Scale to unit screen coordinates We set f to 1 in previous matrix Third row is arbitrary (for now), not used normalized screen coordinates

52 Aspect Ratio 1 w h tan α tan α Non-square screens? Screen: w h pixels Aspect ratio w h Different horizontal angle! h 1 1 w 1 non-square screen normalized screen coordinates

53 y-coord. To Screen Coordinates w/2 0 0 w/2 0 h/2 0 h/ x-coord. w Scale to pixels Third row is arbitrary (for now), not used. h 1 0 h 1 w 1

54 To Screen Coordinates h/2 w/2 tan α 0 0 tan α h/2 h/2 tan α 0 tan α a 1 b Overall Multiple both a = z far + z near z near z far b = 2 z near z far z near z far Additionally: Also scale + shift such that z = z 1 z are in value [0..1] for inputs z [z near, z far ]

55 Summary Projection matrix P = f f Projection & conversion to screen coords P s = w/2 0 0 w/2 0 h/2 0 h/ w h tan α tan α (f = 1) scaling to pixels, upper left origin normalized screen coord s projection matrix

56 Alternative (1) Alternative formulation: Only two steps P s = w/2 0 0 w/2 0 h/2 0 h/ w h tan α tan α scaling to pixels, upper left origin normalized screen coord s & projection matrix Different scale factors (not a focal length) Use two different scale factors f x = 1 w h tan α 2, f y = 1 tan α 2

57 Alternative (2) Another Alternative Formulation w h P s = + w h h/2 0 0 w/2 0 h/2 0 h/ scaling to pixels, upper left origin 1 tan α tan α vertically normalized screen coord s Constant focal length f = 1 tan α 2 Intermediate result not normalized to 1,1 2 & projection matrix

58 Alternatives All three derivations lead to the same result Intermediate results not used all equivalent Product of the 2/3 matrices is the same Intermediate results being used: Some graphics APIs (e.g., OpenGL) do use normalized device coordinates as intermediate OpenGL for pixels to appear on screen: x [ 1,1] y [ 1,1] z [0,1) w z near, z far coupled, so this is the same criterion

59 general camera General Camera object of interest y x z camera in origin, view in z-direction

60 general camera v General Camera c u w object of interest y x z camera in origin, view in z-direction

61 general camera v General Camera Camera coordinate system u, v, w u Origin: c w c y x z Standard coordinates x, y, z = 1 0 0, 0 1 0, camera in origin, view: z-direction

62 c v u w Derivation y x z

63 Derivation y x Same effect: Transform the world with inverse camera transform z c v u w u v w = u v w 1

64 Derivation y x Transform: p u v w 1 p c z c v u w u v w = u v w 1

65 Derivation y x Transform: u p v w u, v, w orthogonal! p c z c v u w u v w = u v w 1

66 General Camera general camera u y c x r z v Camera coordinate system u, r, v Origin: c Transform: u p v w (p c) Standard coordinates x, y, z = 1 0 0, 0 1 0, camera in origin, view: z-direction

67 general camera y c x u r z General Camera v Camera coordinate system u, r, v Origin: c Homogeneous: p u v c w c = p u v w c

68 Summary Projection (screen coord s) P s = h/2 0 0 w/2 0 h/2 0 h/ Add View Matrix 1 tan α tan α (f = 1) Benefit: Still only one overall 4 4 matrix to multiply with! P s u v c w

69 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

70 Visibility Algorithms BASIC CORE basic topics study completely core topics important

71 Rasterization Two Rendering Pipelines Project all triangles to the screen Rasterize them (convert to pixels) Determine visibility Apply shading (compute color) Raytracing Iterate over all pixels Determine visible triangle Compute shading, color pixel

72 Triangle / Polygon Rasterization Observations Straight lines remain straight! Triangles mapped to triangles After Perspective Projection Polygons to polyogns

73 3D Scene Rasterization Visiblity preprocessing or during rasterization Projection Visibility Rasterization

74 Raytracing 3D Scene

75 Comparison Rasterization FOR (each triangle) { } compute pixels covered ( fragments ) FOR (all fragments) { } fragment visible? IF (visible) { } shade fragment write color Raytracing FOR (each pixel) { compute visible triangle IF (found) { shade fragment write color } }

76 Focus for now: Rasterization Rasterization (Raytracing covered later) Two main algorithms Painter s algorithm (old) Simple version Correct version z-buffer algorithm Dominant real-time method today

77 Painter s Algorithm CORE core topics important

78 Painters Algorithm Painter s Algorithm Sort primitives back-to-front Draw with overwrite Drawbacks Slowish O(n log n) for n primitives Millions per second Wrong Not guaranteed to always work

79 Counter Example Correct Algorithm Need to cut primitives Several strategies Notable: BSP Algorithm in Quake Old graphics textbooks list many variants No need for us to go deeper

80 z-buffer Algorithm BASIC basic topics study completely

81 Algorithm Store depth value for each pixel z-buffer Algorithm Initialize to MAX_FLOAT Rasterize all primitives Compute fragment depth & color Do not overwrite if fragment is farer away than the one stored the one in the buffer color depth

82 Advantages Extremely simple Discussion: z-buffer Versatile only primitive rasterization required Very fast GeForce 2 Ultra: 2GPixel /sec (release year: 2000) GeForce 700 GTX Titan: 35 GPixel / sec (release year: 2013)

83 Disadvantages Discussion: z-buffer Extra memory required This was a serious in obstacle back then... Invented 39 years ago (1974; Catmull / Straßer) Only pixel resolution Need painter s algorithm for certain vector graphics computations No transparency This is a real problem for 3D games / interactive media Often fall-back to sorting Solution: A-Buffer, but no hardware support

84 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

85 Shading Models CORE core topics important

86 Reflectance Models mirror glossy surface diffuse surface

87 Interaction with Surfaces Viewer v n Light l object surface Local Shading Model Single point light source Shading model / material model Formalization: BRDF Input: light vector l = pos light pos object Input: view vector v = pos camera pos object Input: surface normal n (orthogonal to surface) Output: color (RGB)

88 Interaction with Surfaces Viewer v n Light l object surface General scenario Multiple light sources? Light is linear Multiple light sources: add up contributions Double light strength double light output Formalization: BRDF

89 Simplify notation Remark Define component-wise vector product x 1 y 1 x 1 y 1 x y = x 2 x 3 y 2 y 3 x 2 y 2 x 3 y 3 No fixed convention in literature The symbol only used in these lecture slides!

90 Lighting Calculations Remark Need to perform calculations for r, g, b-channels Often: output r = light r material r function v, l, n output g = light g material g function(v, l, n) output b = light b material b function(v, l, n) Shorter output = light_strength material function v, l, n

91 Area Light Sources Area Light Sources Integrate over area In practice often: Sample with many point-light sources Add-up contributions n light sources 1 n intensity v n l

92 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

93 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

94 Diffuse ( Lambertian ) Surfaces n surface normal n θ n θ Equation (set to zero if negative) c ~ cos θ Less light received at flat angles c = c r c l cos θ light color surface color c intensity (scalar) c color (RGB, R 3 ) c r surface color (RGB) c l light color (RGB)

95 Diffuse ( Lambertian ) Surfaces n surface normal n θ n θ Equation c ~ cos θ Less light received at flat angles Attenuation: (point lights) 1 dist 2 c = c r c l cos θ 1 dist 2 light color surface color c intensity (scalar) c color (RGB, R 3 ) c r surface color (RGB) c l light color (RGB) 1 dist 2

96 Diffuse Reflection Diffuse Reflection Very rough surface microstructure Incoming light is scattered in all directions uniformly Diffuse surface (material) Lambertian surface (material)

97 Surface Normal? What is a surface normal? Tangent space: Plane approximation at a point x S Normal vector: Perpendicular to that plane Oriented surfaces: Pointing outwards (by convention) Orientation defined only for closed solids S tangent space surface normal n x R 3 point x

98 Triangles p 2 n p 3 Single Triangle Parametric equation p 1 p 1 + λ p 2 p 1 + μ p 3 p 1 λ, μ R Tangent space: the plane itself Normal vector p 2 p 1 p 3 p 1 Orientation convention: p 1, p 2, p 3 oriented counter-clockwise Length: Any positive multiple works (often n = 1)

99 Triangle Meshes Smooth Triangle Meshes Store three different vertex normals E.g., from original surface (if known) Heuristic: Average neighboring triangle normals

100 Lambertian Surfaces n surface normal n θ n θ Equation c = c r c l cos θ = c r c l n, l n light direction normal vector (assuming: n = l = 1) l

101 Lambertian Bunny Face Normals Interpolated Normals

102 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

103 Problem Ambient Reflection Shadows are pure black Realistically, they should be gray Some light should bounce around... Solution: Add constant c = c a c a Not very realistic Need global light transport simulation for realistic results ambient light color surface color

104 Ambient Bunny Pure Lambertian Mixed with Ambient Light

105 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

106 Perfect Reflection Perfect Reflection Rays are perfectly reflected on surface Reflection about surface normal r = 2 n, l n l + l, n = 1 r n l l arbitrary

107 Perfect Reflection Difficult to compute Need to match camera and light emitter More later: Recursive raytracing Right image: Environment mapping Silver Bunny Reflective Bunny (Interpolated Normals)

108 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

109 Glossy Reflection Glossy Reflection Imperfect mirror Semi-rough surface Various models

110 Phong Illumination Model Traditional Model: Phong Model Physically incorrect (e.g.: energy conservation not guaranteed) But looks ok Always looks like plastic On the other hand, our world is full of plastic...

111 How does it work? Phong Model: Specular (glossy) part: (high-) light color c = c p c l r r, v v cos r,v Ambient part: c = c r c a Diffuse part: c = c r c l n, l Add all terms together p 1,2 1 0,8 0,6 0,4 0,2 0 Phong Exponents p=1 p=2 p=5 p=10 p=50 p=100 v r n l

112 Blinn-Phong Blinn-Phong Model: Specular (glossy) part: c = c p c l h h, n n p v h n l cos h,n Half-angle direction h = 1 l + v 2 l v In the plane: h, n h Approximation in 3D n = 1 2 r r, v v

113 Phong+Diffuse+Ambient Bunny Blinn-Phong Bunny Interpolated Normals

114 Phong+Diffuse+Ambient Bunny Blinn-Phong Bunny Interpolated Normals

115 Cook-Torrance Model Physically-Motivated Model D Infinitesimal micro-facets Characterize by distribution Expected reflection (density) Gaussian, Beckmann, Approximate occlusion term (G) F Fresnel term Model: wave-optics c spec = D G F 4 v, n n, l Interaction of wave with surface under different angles Percentage reflection/refraction F θ = R R 0 1 cos θ 5 cos θ = h, v R 0 ="ratio of refractive indices"

116 Artistic Fresnel Reflection Exponent 4 Approx. Fresnel-Reflection F θ 1 cos θ p unweighted reflection Exponent 5

117 Better Models Phong Bunny Cook-Torrance Model

118 Shading effects Diffuse reflection Ambient reflection Perfect mirrors Glossy reflection Phong / Blinn-Phong (Cook Torrance) Shading Effects Transparency & refraction

119 Transparency Transparency Alpha-blending α = opacity Color + opacity: RGBα Blending Mix in α of front color, keep 1 α of back color % red, 50% green front back c = α c front + 1 α c back Not commutative! (order matters) unless monochrome

120 Refraction Refraction: Snell s Law Materials of different index of refraction Light rays change direction at interfaces Snell s Law sin θ 1 sin θ 2 = n 2 n 1 n 1, n 2 : indices of refraction vacuum: 1.0, air: water: 1.33, glass: θ 2 n θ 1 n n 1 n 2

121 Refraction Reflection Refraction Implementation (raytraced) Not a local shading model Global algorithms: mostly raytracing Various fake approximations for local shading

122 3D Rendering Steps Geometric Model Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

123 Shading Algorithms CORE core topics important

124 Flat Shading Flat Shading constant color per triangle

125 Flat Shading Gouraud Shading Algorithm compute color at vertices, interpolate color for pixels

126 Flat Shading Phong Shading Algorithm interpolate normals for each pixel

127 3D Rendering Steps Geometric Model Next: Advanced Rasterization Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination

128 3D Rendering Steps Geometric Model Global Illum: Keep this for later Perspective Visibility Local Illumination Smooth Shading Simple Shadows Global Illumination