Lighting and Materials

http://graphics.ucsd.edu/~henrik/images/global.html Lighting and Materials Introduction The goal of any graphics rendering app is to simulate light Trying to convince the viewer they are seeing the real world Primary focus is on realism There are models so accurate they reflect all understood properties of light, however we can t use these in real-time Below is a scene lit with local illumination 1

Introduction The goal of any graphics rendering app is to simulate light Trying to convince the viewer they are seeing the real world Primary focus is on realism There are models so accurate they reflect all understood properties of light, however we can t use these in real-time Below is lit with global illumination Introduction To get things to look this good, we are going to make many assumptions and cheat heavily to the interaction of light and matter 2

Introduction To calculate proper lighting, we need 3 things: Sensor Material Bowling Ball Material Light source Brick Material Glass Material Introduction A light is represented by geometric rays; it is the source of illumination in a scene The most common approximations include: Directional light Point light Spotlight 3

Introduction A material is a complete description of the visual properties of a mesh Can include many properties, however we care mainly about textures, shader parameters, and more Each of these objects have different visual properties represented by the material Introduction There are many materials types, each responsible for different effects We will use Phong and Stingray PBS 4

What Light Does Light is emitted from a source Light interacts with objects in the scene and is either: Absorbed Scattered Reflected Light is absorbed by a sensor Only a small fraction of the total light hits the sensor Light Background - Radiometry Radiometry Measuring the physical transmission of light measured as electromagnetic radiation Frequency is related to the energy of a photon Different frequencies make up the entire spectrum of color we see 5

Light Background - Radiometry Output of a light source measured through radiant flux Radiant flux = the number of joules per second emitted (summed energy of all photons in 1 second) 100 watt light bulb We only see a small section of the total photons emitted Light Background - Radiometry Irradiance is the density of radiant flux with respect to an area (usually a surface) Irradiance light flowing into a surface Exitance light flowing out of a surface If the rays of a light source are parallel, the irradiance will not vary with location Same irradiance 6

Light Background - Radiometry Irradiance is the density of radiant flux with respect to an area (usually a surface) Irradiance light flowing into a surface Exitance light flowing out of a surface If the light source is very close the irradiance will vary with distance High irradiance Low irradiance Light Background - Sensor When light interacts with matter it is either: 1. Reflected 2. Scattered 3. Absorbed A small potion of that light will make it to an imaging sensor: Eye rods & cones Digital Camera photo diodes Film dye particles 7

Light Background - Sensor A sensor by itself detects the irradiance (incoming light) value over the surface and produces a color signal To produce this we need: Light proof enclosure Single small aperture to restrict the directionality of light (irradiance to radiance) Lens to focus the light onto the sensor. Light Background - Sensor Our aperture restricts the directionality of light that strikes the sensor Apertures determine how collimated (parallel) the rays are Parallel light will not disperse with distance If light rays diverge, they lead to a blurry image 8

Light Background - Sensor Restricting the directionality of incoming light we are measuring the radiance (density of light flow per area and per incoming direction) Brightness and color are represented in a single ray through an RGB vector with unbounded values. Light Background - Sensor In computer graphics, each sensor measures only a single radiance sample in an instant rather than an average area over an average time (film) A ray is from an object to the sensor Converge at the common point p, which is the center of projection for the perspective transform The detector is replaced with a shader equation evaluation step 9

Light Background - Sensor Each evaluation will compute the radiance along the view vector ray, v This evaluation will occur for each scene fragment generated Light Transport Models Light transport models mathematical models of lightsurface / volume interaction There are two types of models that have been / are being used: 1. Local Illumination used due to simplicity 2. Global Illumination the direction the industry is moving toward, but more complex 10

Light Transport Models https://youtu.be/hlqamnvlibo Types of Lights We approximate real-time lighting in Unity with: Ambient Light Ambient term, independent of view angle, no specific direction. Increase overall brightness Directional Lights Infinite distance away, light rays appear to be parallel. Light has no position, just direction Point Lights Radiates uniformly in all directions, has a position, used inverse square law based on a range value Spot Lights A point light with the rays limited to a coneshaped region. Has a position, used inverse square law based on a range value Area A rectangle in space from one side. Diminishes by the inverse square and only for baked lights 11

Local Illumination Simplest to calculate, computes only direct lighting Direct lighting - light interacts with a single object and proceeds directly to the imaging plane ( no bouncing ) Objects do not affect one another s appearance First model to be used in games, default setup in the fixed functionality pipeline Does not look beyond each vertex ( or each pixel ) Shadows, color bleeding, etc are not directly determined Local Illumination Light is evaluated at each vertex and immediately used to calculate pixel color (1 bounce) Object Occlusion does not matter Local Illumination, calculations are performed at each point with no regard for the rest of the scene 12

Global Illumination Photorealism requires accounting for indirect lighting and object occlusion Light bounces from surface to surface before arriving at the virtual camera Photons may interact with surfaces many times before reaching the viewer We limit this to a set number of bounces Global Illumination Global solutions are needed if you want to simulate: Shadows Reflective surfaces Inter-reflection between objects Caustic effects 13

Local Illumination Global Illumination 14

Ray Tracing https://www.youtube.com/watch?v=woqr0k2ia9a Light Sources Light is represented as a vector Light is emitted from a source and travels through space The simplest: Directional Light Light sources with nearly parallel rays can be generalized to a directional light The sun from Earth s surface can be represented as a directional light 15

Light Sources Light vectors are unit length Consider all vectors from the surface of the object (flip the light s vector) A directional light source has a direction and intensity This intensity is our radiance Color and intensity can be denoted with [R,G,B] vector l = light vector Light Sources Illumination of a surface requires light direction and the surface direction to be compared The dot product between the light vector and the surface normal indicates the final intensity of light entering the surface 16

Light Sources E is the radiance perpendicular to the plane normal E L is the radiance perpendicular to the light vector (max intensity) We clamp the cosine value to non-negative values E L E E = E L cos θ i E = E L max(n l, 0) Light Sources Radiance is additive All incoming photons from n lights sum to the following equation: n E = E LK cos(θ i k ) k=1 n = number of lights E = Total Radiance 17

Light Sources 1. Dot the light and the surface normal to determine the radiance per light 2. Multiply the dot product s value by the light s color 3. Sum the lights for the final light color 4. Multiply component by component for light color and material color Materials The actual appearance of an object depends on the underlying surface characteristics This is modeled using a shader program which uses surface properties to determine interaction with light Every material will have a different set of characteristics 18

Materials Light-matter interactions have two major classifications: 1. Scattering 2. Absorption Scattering when light encounters an optical discontinuity The interface of two substances with different optical properties A break in a crystalline structure A change in density Scattering changes the direction, not the amount of light Materials Absorption happens inside matter, converted into another form of energy (disappears from our scene) Absorption affects the amount of light not the direction 19

Materials The air/model interface is the most important for us to model A surface will scatter light into two directions: 1. Reflection out of the surface 2. Refraction/transmission into the surface Materials The properties of an object depend on the structure of the object on a microscopic scale We can not expect to model this kind of detail into an object Bidirectional Reflectance Distribution Function (BRDF) is a method used to describe the scattering of light directly off the surface (no subsurface scattering) 20

Materials The properties of an object depend on the structure of the object on a microscopic scale We can not expect to model this kind of detail into an object Bidirectional Reflectance Distribution Function (BRDF) is a method used to describe the scattering of light directly off the surface (no subsurface scattering) Materials The surface material will need to take into consideration the characteristics of the surface, the size of the micro-scale geometry and the wavelength of light Object can appear darker at some angles 21

Materials Reflected light is approximated by the specular term Light that is reflected will have the color of the light source Transmitted light is wrapped into a diffuse term (light diffused through the surface) Materials We handle the specular and diffuse components separately because they each depend on different inputs For simplicity the diffuse term is direction independent C ambient + C spec + C diffuse = C overall The specular component is dependent on the surface smoothness and viewer The reflected ray s tightness will relate to surface irregularities 22

Material & Light Interaction Light and materials interact partly on their color Light source R: 1.0 G: 1.0 B: 1.0 Material R: 1.0 G: 0.1 B: 0.4 Final Pixel Color R: 1.0 * 1.0 = 1.0 G: 1.0 * 0.1 = 0.1 B: 1.0 * 0.4 = 0.4 Material & Light Interaction Light and materials interact partly on their color Light source R: 0.0 G: 0.0 B: 1.0 Material R: 1.0 G: 0.1 B: 0.4 Final Pixel Color R: 0.0 * 1.0 = 0.0 G: 0.0 * 0.1 = 0.0 B: 1.0 * 0.4 = 0.4 23

Materials - Shininess Phong Lighting Model Phong - Most common local illumination model Sum of three distinct terms: Ambient + Diffuse + Specular = Light Reflected Ambient overall lighting level in a scene, gross approximation of indirect bounced light Diffuse reflected light uniformly reflected in all directions (matte surface reflectance) Specular bright highlights seen when viewing a glossy surface, when the viewing angle is closely aligned with light directly reflected from a light source 24

Phong Lighting Model Calculate the total intensity at a point I : I p = k a i a + i k d L i N i d + k s (R i V) α i s I pv A N k a, k d, k sα i a, i d, i sli R i = intensity of reflected light at a point = viewing direction vector = ambient light intensity = surface normal = material color = specular exponent (shininess) = light s color = light direction vector = reflection vector Phone Lighting Model Diffuse Intensity From OpenGL 4.0 Shading Language Cookbook I d = i d k d (തL ഥN) Specular Intensity I s = i s k s തR തV α r Ambient Intensity I a = i a k a The K terms are reflectivity terms composed of vector3 s, which denote the color of the surface 25

Phong Lighting Model Reflection vector can be calculated: From OpenGL 4.0 Shading Language Cookbook A vector can be expressed as the sum of its tangential and normal components L = L T + L N L N = N L N R = L N L T R = L N L L N R = 2L N L R = 2 N L N L L N (unit) L N R L T -L T Phong Lighting Model From OpenGL 4.0 Shading Language Cookbook Phong lighting is referred to as Gouraud lighting when applied per vertex If applied per fragment, we get nicer results at the expense of MANY MORE CALCULATIONS 26

Blinn-Phong Lighting Model Jim Blinn proposed a variation of the Phong lighting model with a faster method for calculating specular reflection Use a halfway vector H between the view and light vector R V α N H α Half vector determined by suming the Light and View direction and normalizing Faster to calculate and in some cases more accurate than the Phong model Half Vector vs Reflection Vector The reflection vector is based on Fresnel Reflection The half angle has a physical interpretation that only those microfacet details that bend light directly from the source the viewer will be seen in the specular highlight 27

Half Vector vs Reflection Vector Half Vector highlights are elongated at glancing angles and are more dependent upon the view direction Reflection Half BRDF The Blinn and Phong lighting model represent special cases in a general reflectance model called: bidirectional reflectance distribution function (BRDF) A BRDF calculates the ratio of outgoing radiance along a given direction to the incoming irradiance along an incident light ray Visualized as a hemispherical plot Distance from origin is the intensity of light 28

BRDF For diffuse, the value is the same in all directions (does not account for the direction of the viewer V N L BRDF The specular term is dependent on the view and light direction Produces a hotspot when the view and light are closely aligned Falls off quickly based on the exponent coefficient R V N L 29

PBR Physically based rendering (PBR) is what we use within Unity and Unreal engine Has become a standard practice for most games With increasing computational power we can now approximate light in better ways Create a new material and by default you will get a Standard shader, which uses PBR https://marmoset.co/posts/basic-theory-of-physically-based-rendering/ PBR Instead of approximating derived qualities such as specular highlight, we let the light act in a physically based way We adhere to Energy Conservation: Incoming light = Reflectance + Diffusion The more reflective a surface is, the less diffuse it is: 30

PBR - Metals Electrically conductive objects usually are very reflective Some conductors have reflectivity that varies along the visible spectrum and will tint the color Conductors usually absorb any light rather than scatter it Because metals and non-metals have these opposing characteristics, it is one of the ways PBR is partially defined PBR Fresnel Effect Reflectivity varies based on the angle that it occurs at (straight on vs grazing) Object reflections are brighter at the edges Any smooth object can become a mirror at a grazing angle Allow the base of the Fresnel reflection to be handled by the artist and let the computer calculate the gradient Remember this is only for smooth objects! 31

PBR Fresnel Effect PBR Microsurface Detail Microsurfaces cause rays of light to become diffuse, softening the surface and making reflections disappear Quantify this term as Roughness or Glossiness Microsurfaces = Roughness 32

PBR Reflections Increasing roughness means less light gets to the camera Therefore with increasing roughness the object should appear dimmer The rougher the object, the duller the material PBR Reflections 33

PBR Reflections PBR Reflections 34

Unity Lighting In Unity, we can use a mixture of static and dynamic lighting (baked vs. realtime) By default, lights are set to realtime, they are calculated every frame and movement of objects / lights will occur every frame Realtime lighting does not use global illumination, that requires baking the scene Unity Lighting Baking only affects static objects and stores lighting calculations in a texture that is overlaid on the scene Baked lighting can look awesome, but can not be modified at runtime! 35

Unity Lighting To react to changing lighting conditions, Unity includes Precomputed Realtime GI lighting Simplifies geometry into clusters and calculates the relationships between clusters to see how bounce lighting interacts between them Provides the best of both worlds Unity Lighting Do note that baking and precomputed realtime GI are separate systems Using both will take as much time as calculating each separately Depending on your hardware, you may only want to use 1 system, or even both at the same time 36

Unity Lighting Systems are controlled through the Lighting window Can be controlled per-light in the inspector, just make sure the lighting mode is enabled Unity Lighting Realtime Will use realtime lighting, with no bouncing Baked will precompute lighting on objects that are assigned as static lighting in the static drop down Mixed Will bake to static objects but also perform realtime lighting on non-static objects 37

Unity Lighting Unless disabled, precomputing is automatically performed while you work In order for any pre-compute to begin, at least 1 object must be static in the scene The data cache is stored outside your project, and so will need to be rebuild if moved to another computer Unity Lighting Reflection probes are used to sample the scene and generate cubemaps of the environment Can be either realtime or baked Each probe causes the scene to be rendered an extra 6 times, once for each side of the cube Objects must be marked as reflection probe static from the static drop-down menu to appear in the cube map Realtime probes render everything regardless of static settings 38

Unity Lighting Ambient light is controlled in the scene through the lighting windows ambient source field Ambient light can be either a color, gradient or skybox Unity Lighting We can also create Emissive light sources through the material properties of our objects 39

Unity Lighting Light probes allow dynamic objects to pick up color information from the precomputed world Performed by using the nearest light probes and averaging their values 40