Normal Maps and Cube Maps What are they and what do they mean?
What s the Point of All This? Q: What re we doing?
What s the Point of All This? Q: What re we doing? A: Making a game that looks good
What s the Point of All This? Q: What re we doing? A: Making a game that looks good Q: What re we really concerned with regards to looking good?
What s the Point of All This? Q: What re we doing? A: Making a game that looks good Q: What re we really concerned with regards to looking good? A: Essentially, whatever the players perceive in their minds of light s interaction with the world such that it is believable
What is Global Illumination? All sorts of light bouncing around in the scene
What is Global Illumination? diffuse specular
What is Global Illumination? and light finally reaches the viewer
How do People See Things? We see light via a grid
Then What is a Picture? An array of contiguous grid points forms a map
Is That It? Are we limited to just a picture for a map? No, there s much more to this!
Is That It? A map can be thought of as just some arbitrary set of samples! However, in graphics, we use 2D maps to represent some mathematical quantities of surface characteristics (e.g. textures of surfaces, surface normals, surface displacement, surface of the skydome, or even the surfaces of a cube)
What s the Point of All This? Turns out that the most important feature in the average image comes from direct lighting of surfaces I.e. the path where the light bounces off of one surface and enters the eye directly We realized this, so we do geometry lighting, and used texture maps to make the lighting results look even better
It s All in the Details What if we want the best possible results for surface lighting? For best looking results: keep everything that we started with (e.g. all the polygons, normals, UVs, details heck, why not even the original spline surfaces?) But:
Recall
It s All in the Details Too much memory requirements Too much processing requirements I.e. too expensive for us to do and maintain a reasonable framerate Worst of all, it s not even necessary for good-looking, believable output (from today s standards)
Alternatives? Q: What re we really concerned about in terms of local surface details? A: How light reflects off surfaces
Some tessellation Alternatives?
Focus in on a spot Alternatives?
Surface is flat on a per-triangle basis Alternatives?
Alternatives? If one thinks of light as a ball, then: We re mainly concerned with how light bounces off of surface details In other words, how the individual, flat elements of the surface appear to the viewer
Alternatives?
Specular Reflection This is just mirror reflection
Surface Facing Mathematically, the surface facing is described by the normal it s just a direction in space that is perpendicular to the facing of the surface It s really just the direction that the surface is facing
Per-Vertex Lighting How do we work with normals? We use normals on a per-vertex level to light triangles We then interpolate the vertex lighting results to generate smooth lighting for the fragments This is natural because the geometry approximates a smooth surface
Interpolated Vertex Normals However, a problem arises: high frequency lighting isn t being captured we only get correct values for the end points, not the in-between fragments Instead, we interpolate the per-vertex normals, not the lighting color Do lighting on the interpolated per-vertex normals
One Step Further Well, now that we re interpolating normals across fragments, we may as well take this one step further We want normals that we can be modified on a per-fragment level
One Step Further So, how do we do this without using a ton of extra vertices to fill in the gaps? We use a set of values that represent normals and map those values onto a surface a map of normals, or, a normal map They re applied (most of the time) just like regular texture maps
One Step Further Look up a normal map just like a texture
One Step Further When we render a fragment of a triangle, we use the UV coordinates to look up the normal map That lookup result is what we use to calculate the normal to be at that point To the artists, this means adjusting a normal is equivalent to reorienting the surface facing of that particular fragment
Adjusting the Normal
Example
Example
Example
Example
Example
Similarities? Many may have noticed that normal maps resemble some other techniques used in modeling surface details Yes, normal maps are intimately related to bump maps, displacement maps, parallax maps, etc They re all just ways to describe surface geometry!
Disclaimer In practice, to avoid seams and weird transitions, normal maps usually assume that the underlying geometry is continuous in its normals As in, there is no sudden transition in the normals Discontinuities signify hard edges, where the geometry is meant to be not smooth
Disclaimer Two ways to model a cube: 1) Faces with normals that are constant in the direction that they face 2) Faces with normals that are continuous across all adjacent surfaces Usually, only the 1 st case is used for a cube
Disclaimer Normal maps are perturbations to the underlying geometry s interpolated normals As in, the normal map results are used to offset the interpolated per-vertex normals MLee: It s important to note that nondegenerate geometry shouldn t have degenerate UVs, or we can t generate a correct UV basis
More Goodness All this time, the examples only focused on direct lighting from local light sources What does a fragment really see?
More Goodness Photons don t just come from a single light source
More Goodness They come from every direction on the hemisphere of the fragment we re rendering But the problem is, it s way too expensive (if at all possible) to model all the interreflections entering through that hemishpere centered on that fragment
The Simple Solution Is there a way to approximate environment lighting? Well, what have we been using all this time in terms of environmental lighting? Lighting = Specular + Diffuse + Ambient
The Simple Solution What is ambient lighting? It is just a very simple form of environment lighting We re basically saying that a light source contributes a certain amount of inter-reflective lighting to the scene Of course, it can also be used for other things such as setting the mood of a level
Environment Lighting To come up with a better looking environment lighting model, in addition to local lighting, we will break down the problem of capturing light coming from the environment into two parts: Light from objects that are close by Light from objects that are far away (*No real-time inter-reflections, sorry)
Why This Way? Humans perceive the world through perspective projection we experience foreshortening and parallax Objects further away feel like they move less in our field of view than objects that are closer (e.g. just like the backgrounds of old-school 2D side scrollers)
Why This Way? Very little movement for the clouds! I think
So What? We can exploit this lack of difference/movement between vantage points We will just assume that distant objects are all so far away that they never move relative to each other, even when the viewer moves about
Why This Way? Proof : That s 20 feet away between shots
And the Result? Light coming from far away looks the same no matter how we move I.e. the specular reflection of incoming light off of far away objects look the same even if we strafed around (as long as we maintain our facing) So instead of being lit by just the ambient term of a point light source, now we re lit by some environment light source that s dependent on the reflection direction
Cube Environment Maps Think of a specular cube environment map as a sky dome in the game We think of the surface of our near-by object as a mirror to calculate the specular direction of the reflection Then use the direction of the reflection to look up the cube environment map (where the reflection intersects the cube map)
Cube Environment Maps The result is used as the specular lighting for the fragment
More Cube Environment Maps This technique isn t limited to specular lighting We can use the same technique for diffuse/irradiance environment maps One way is to pre-sum the hemisphere that any point on the surface can see, and store that result into where the normal points at Therefore, look up by normal, not by reflection direction
More Cube Environment Maps direct lighting indirect lighting
Summary We want to model as much of a believable environment as possible Use normal maps for surface detail approximation Use environment maps for environment lighting approximation
Credits The countless people whose work of art (e.g. pictures) I used
Questions?