TEXTURE MAPPING DVA338 Computer Graphics Thomas Larsson, Afshin Ameri
OVERVIEW Motivation Texture Mapping Coordinate Mapping (2D, 3D) Perspective Correct Interpolation Texture Filtering Mip-mapping Anisotropic Filtering Special Effects Bump Mapping Displacement Mapping Shadow Mapping Environment Mapping
MOTIVATION
Phong Shading seems nice, much better than the Flat Shading or even the Gouraud Shading, But
Uniform reflectance (color) Complex reflection properties More detailed geometry Wallpapering or gift-wrapping a simple geometry Textures Picture from: yourpalmark.com Surface detail, Reflections, Shadows, etc. can it do this?
It s all about details Picture from: www.3drender.com/jbirn/productions.html
MOTIVATION Real surfaces are often very complex. We do not want/can represent all the fine variation in geometry. We need other ways to add surface detail. We can increase the apparent complexity of simple geometry by wallpapering or gift-wrapping with stretchy paper Special effects: Bump mapping Shadow mapping Environment Mapping
TEXTURE MAPPING
A Texture Map is a function or pixel-based image that is used to store detailed information about a surface. Individual pixels of a texture map are called Texels (texture elements). Texture Mapping is the process of applying texture map data onto the surface. Texture maps can be 2D (image textures) or 3D (volume textures or solid textures). We should define mapping functions on how to map a point on the surface to the image or function. Mapping functions maybe different in case of 2D or 3D texture maps.
TEXTURE MAPPING 2D Texture Mapping
2D TEXTURE MAPPING Texture Map A normal 2D space Coordinate system: lower left (0,0), upper right (1,1) regardless of the image size Usual texture sizes are 128x128, 256x256, 512x512, 1024x1024 Information on textels may be in various formats: RGB, RGBA
2D TEXTURE MAPPING Texture Map Specify a texture coordinate (u,v) at each vertex Canonical texture coordinates: (0,0) (1,1) The image itself ranges from 0.0 to 1.0, independent of the actual pixel resolution However, this doesn t mean that texture coordinates have to be limited to that range The texture coordinates are used to look-up values in the actual texture map E.g.: gettexel(tmap, u, v) Note: The individual pixels of the texture map are called texels (texture elements)
2D TEXTURE MAPPING Image Source: Tschmits, published under CC-BY- AA
2D TEXTURE MAPPING Mapping Process Assigning texture coordinates (u,v) to vertices. This is normally predefined. The texture coordinates are interpolated across the triangle in the scan conversion process. Once we have a texture coordinate for the pixel, we perform some sort of texture lookup. This texture color is then usually blended with the interpolated RGB color to generate the final pixel color. (0.5,1) gettexel (1,1) (0,0) scan line Texture coordinates Triangle primitive (1,0) (0,0) Texture map
2D TEXTURE MAPPING Mapping Process The Math involved So we have (u,v) of the triangle vertices and we need the (u,v) for the point being rasterized. It can simply be calculated using barycentric coordinates and we will have: u p = u a + β(u b -u a ) + γ(u c -u a ) v p = v a + β(v b -v a ) + γ(v c -v a ) u p and v p are then looked up in the original texture to get the texel value. Of course, we do not do the calculations in code and simply leave it to OpenGL. (0.5,1) gettexel (1,1) (0,0) scan line Texture coordinates Triangle primitive (1,0) (0,0) Texture map
EXAMPLE: TEXTURED CUBE A texture mapped 3D cube model Texture map, size 512 x 512 What texture coordinates have been used at the polygon corners in this example? 16
TILING We can define various tiling or wrapping rules to determine what happens when we go outside of the 0 1 range Clamp, repeat, mirror, Mode can be set independently in x and y (1, 1) (0, 0) Original texture clamp repeat mirror 17
TEXTURE MAPPING & LIGHTING Texture mapping can be used to alter various constants in the illumination equation The simplest examples: Combine Phong Shading and Texture mapping Don t use a separate material color at all (or set to (1,1,1)) 18
2D TEXTURE MAPPING What s wrong with this box? Affine Texture Mapping This one seems much nicer! Perspective Correct Texture Mapping Pictures from: www.gamedev.net
2D TEXTURE MAPPING Perspective Correct Texture Mapping The texture may warp and stretch within the triangle as the viewing angle changes, an effect called texture swimming. Observation: a straight line of regularly spaced points in 3D space maps to a straight line of irregularly spaced points in screen space. Linear interpolation of texture coordinates in screen space is wrong, because homogenous division has distorted barycentric coordinates of the triangle. Depth of the pixels should also be involved in order to get a Perspective Correct Mapping. Original texture Wrong Correct
2D TEXTURE MAPPING Perspective Correct Texture Mapping At screen space 1/z, u/z, v/z are linear and interpolate correctly. We can interpolate u/z and v/z to get correct interpolated values u/z and v/z need to be multiplied by z in order to give us (u,v) z is also changing and needs to be correctly interpolated. We can interpolate 1/z. Dividing u/z and v/z by 1/z will give us the correct (u,v). BUT, we do not know z anymore Instead we know that w 1/z SO, we interpolate uw, vw and w, then we can have u=uw/w and v=vw/w
2D TEXTURE MAPPING Texture Filtering What is wrong with this image? It s a sample of a problem called Aliasing. It occurs because one texel does not map to one pixel most of the times. If the pixel is larger than a texel we should somehow get an average color from those texels (Minification). If pixel is smaller than a texel we should interpolate between the texels (Magnification). Pictures from: www.phnet.fi/public/realsoft/ Does this image need minification or magnification methods to look better? If you wanted to give an example of the other problem, what would it be? You may have heard of a term called Undersampling, what is it?
2D TEXTURE MAPPING Texture Filtering Magnification Picture work of Sunchirp: http://sunchirp.blogspot.com/
2D TEXTURE MAPPING Texture Filtering Magnification Point Sampling (Nearest Neighbor): Simply pick the nearest sample. Bilinear Sampling: A bilinear blend of the four nearest samples. Bicubic Sampling: Performs a smoother bicubic blend of a 4x4 grid of texels. (u, v) texels
2D TEXTURE MAPPING Texture Filtering Minification Blend all the texels within the area to get a single final color. Very expensive, mostly due to memory access costs. Several proposed methods: Antialiasing: Prefiltering, Supersampling Mipmapping Summed-area table texturing Elliptical Weighted Averaging... Picture from:: www.tomshardware.com
2D TEXTURE MAPPING Texture Filtering Prefiltering A pixel is treated like an area on the surface of the object (Catmull, 1978). The area is mapped to the texture map. An average color of the texels involved is calculated. Prefiltering Source: Teaching Texture Mapping Visually by Rosalee Wolfe / DePaul University / wolfe@cs.depaul.edu
2D TEXTURE MAPPING Texture Filtering Supersampling Each corner of the pixel is mapped to the texture. The color from the 4 mapped points is used to calculate the average. Supersampling Source: Teaching Texture Mapping Visually by Rosalee Wolfe / DePaul University / wolfe@cs.depaul.edu
2D TEXTURE MAPPING Texture Filtering Prefiltering and Supersampling in Action Prefiltering Aliasing Supersampling Source: Teaching Texture Mapping Visually by Rosalee Wolfe / DePaul University / wolfe@cs.depaul.edu
2D TEXTURE MAPPING Texture Filtering Mipmapping The method of choice for most graphics hardware. For each texture, several scaled down versions of it are pre-calculated and stored. These are called Mipmaps. Mipmaps are usually half of the size of the previous texture. Their size is usually a power of 2. They have the same size both in x and y directions (they are square). i.e. if the original texture is 512x512 there will be 8 mipmaps for it sized at: 256x256, 128x128, 64x64, 32x32, 16x16, 8x8, 4x4, 2x2, 1x1 This does not add too much texture data since: 1/4 + 1/16 + 1/64 +... = 1/3 The size of the mipmap to be used will be decided by the distance of the object from the camera.
2D TEXTURE MAPPING Texture Filtering Mipmapping Source: mobilarena.hu
2D TEXTURE MAPPING Texture Filtering Mipmapping Source: www.tomshardware.com
2D TEXTURE MAPPING Texture Filtering Anisotropic Filtering Source: ixbtlabs.com
2D TEXTURE MAPPING Texture Filtering Anisotropic Filtering Source: ixbtlabs.com
2D TEXTURE MAPPING Texture Filtering Anisotropic Filtering Source: ixbtlabs.com
2D TEXTURE MAPPING Texture Filtering Anisotropic Filtering When viewing objects from tight angles, the axis will not have the same texture frequency. Mipmapping uses isotropic textures (square) for sampling. Loss of detail in one axis causes the loss in another axis. Anisotropic filtering, solves the problem by sampling anisotropic samples of the textures (128x64, 128x32, ). True anisotropic filtering can be done by sampling the correct trapezoid shape of a pixel according to perspective. It gives very good results, but it is very expensive in memory bandwidth usage and to some extent calculations.
TEXTURE MAPPING 2D Texture Mapping Special Effects
2D TEXTURE MAPPING SPECIAL EFFECTS We can use texture to store other information than color. Textures can be blended with each other or used to change different constants in illumination equations. Some examples are: Bump Mapping Displacement Mapping Shadow Mapping Environment Mapping
2D TEXTURE MAPPING SPECIAL EFFETCS Bump Mapping Dark and light areas on a surface helps us to determine unsmooth areas on an object. By modifying the normal (N) in illumination equations we can create unsmooth areas on the object. (First proposed by Blinn at 1978) This requires per-pixel shading. A texture map is used to store the amount of normal disturbance at each point. Some variations: Normal Mapping, Parallax Mapping. Image Source: Wikipedia
2D TEXTURE MAPPING SPECIAL EFFETCS Bump Mapping
2D TEXTURE MAPPING SPECIAL EFFETCS Bump Mapping What is wrong in this image? No change in object silhouette. Wrong shadows. Image Source: Gdallimore, Wikipedia
2D TEXTURE MAPPING SPECIAL EFFETCS Displacement Mapping An alternative to bump mapping and other similar methods. Using a texture to actually displace the points on surface. Normally done in the direction of the normal. Assuming a displacement texture d: p = p +d(p)*n This changes the vertices, so it needs to happen in geometry processing: We should have lots of tiny polygons to the displacement can happen smoothly. Or, we should use some kind of adaptive tessellation strategy. Recently added to DirectX or OpenGL. Note that the displacement should be defined before the visibility is defined. Image Source: Wikipedia
2D TEXTURE MAPPING SPECIAL EFFETCS Displacement Mapping Bump Mapping Displacement Mapping Image Source: thoronir.net/tutorials Which one is created by Bump Mapping and which one by Displacement Mapping?
2D TEXTURE MAPPING SPECIAL EFFETCS Environment Mapping (Reflection Mapping) A hack to get reflection effect on objects. How it works? Render the scene from the center of a cube in each face direction. Store each result for each face in a texture. Apply these textures on the object when rendering the scene from actual viewpoint. It is an approximation of the reflections on a reflective surface: Reflection on nearby objects is misplaced. No self-reflection are possible. Image Modeled by: Steve van der Burg
2D TEXTURE MAPPING SPECIAL EFFETCS Environment Mapping 6179 Raytrace sec Env. 543 Map sec Image Modeled by: Steve van der Burg
2D TEXTURE MAPPING SPECIAL EFFETCS Shadow Mapping (Projective Shadowing) Invented by Lance Williams in 1978. An image-space algorithm: Independent of scene complexity Ideal for shadows from spotlights Works with anything that can be rasterized Easy to implement Must deal with aliasing artifacts Well-known shadowing algorithm. Observation: If a point is visible both from the light s position and camera, then it is not in the shadow. Therefore we can use the Z-Buffer to determine shadows.
2D TEXTURE MAPPING SPECIAL EFFETCS Shadow Mapping Render the scene from light s point of view. Store the Z-Buffer. When shading an object, transform the point to light s coordinate system. Compare the depths. If the current depth is greater than shadow map depth, then the point is in shadow.
ALGORITHM OVERVIEW Render scene from the light s point of view Store depth of each pixel When shading a surface: Transform the surface point into light s coordinate system Compare current surface depth against depth stored in shadow map If surface depth > shadow map lookup, the pixel is in shadow; otherwise the pixel is lit 47
EXAMPLE: SHADOW MAPPING Pass 1 Pass 2 Light source Z-buffer of light s point of view Camera view with shadows Light s point of view (Images by Mark J. Kilgard, NVIDIA ) Camera view without shadows 48
2D TEXTURE MAPPING SPECIAL EFFETCS Shadow Mapping - Visualization Camera view - Unshadowed Light source view Shadow Map Shadow map projected to camera view Light s planar distance projected to camera view Shadowed scene after depth test Source: Shadow Mapping and Shadow Volumes by Andrew V. Nealen in DevMaster.net
2D TEXTURE MAPPING SPECIAL EFFETCS Shadow Mapping Problems Aliasing Problems Biasing Problems Several Proposed Solutions: Filtering Techniques Polygon offset for depth comparisons... Source: Shadow Mapping and Shadow Volumes by Andrew V. Nealen in DevMaster.net
SHADOW MAPPING PAPERS Lance Williams, Casting Curved Shadows on Curved Surfaces, SIGGRAPH 78 William Reeves, David Salesin, and Robert Cook, Rendering antialiased shadows with depth maps, SIGGRAPH 87 Mark Segal, et. al., Fast Shadows and Lighting Effects Using Texture Mapping, SIGGRAPH 92 51
TEXTURE COORDINATE ASSIGNMENT How can we assign texture coordinates to vertices? Type in the numbers by hand (easy for the cube) Use some sort of automatic projection during modeling process Some modeling programs support painting of texture coordinates Sometimes texture coordinates are assigned on-the-fly during rendering to achieve dynamic real-time effects 52
TEXTURE COORDINATE MAPPINGS Planar Mapping let the texture fill all of space with the same color for all z-values, that is (u, v) = (x, y) Cylindrical Mapping Use the same color for all pixels with the same angle (u, v) = (θ / 2π, y) Spherical Mapping 53
EXAMPLE: COORDINATE MAPPINGS 54
TEXTURE MAPPING 3D Texture Mapping
3D TEXTURE MAPPING Mapping from 3D space to 3D space Easier than 2D mapping. Uses a simple 3D to 3D function. No need to stretch and wrap. Object looks like it has been carved out of a material. No real texture as we are used to. Mostly computational procedures. Ideal for wood, marble, etc. 2D Mapping 3D Mapping Source: www.vrarchitect.net
3D TEXTURE MAPPING The stripes on the left teapot are generated with a simple function. It maps the Z value of a point to a color. If the integer part of Z is even we choose red, otherwise we choose white. What is the function for the other two teapots? Which coordinate values have been used for texturing these teapots? How the function works? Source: Teaching Texture Mapping Visually by Rosalee Wolfe / DePaul University / wolfe@cs.depaul.edu
3D TEXTURE MAPPING Perlin Noise Regular patterns are not interesting enough. Most of the textures have some level of randomness in them. A good random noise function should have the following properties (Perlin 1985): Known range Stationary Band limited Isotropic Image Source: Wikipedia
3D TEXTURE MAPPING Perlin Noise Using Perlin noise to create a 3D wood texture (Peachy 1985) Source: Teaching Texture Mapping Visually by Rosalee Wolfe / DePaul University / wolfe@cs.depaul.edu
3D TEXTURE MAPPING Perlin Noise Perlin noise is a powerful tool. All these images are created by Terragen software only by random noise. Created by: Jay Testerman Source: Planetside Software. Terragen Image Gallery.
3D TEXTURE MAPPING Perlin Noise Perlin noise is a powerful tool. All these images are created by Terragen software only by random noise. Created by: Hannes Janetzko Source: Planetside Software. Terragen Image Gallery.
3D TEXTURE MAPPING Perlin Noise Perlin noise is a powerful tool. All these images are created by Terragen software only by random noise. Source: Planetside Software. Terragen Image Gallery.
TEXTURE MAPPING IN OPENGL OpenGL supports much of the things we ve discussed: 1D, 2D, and 3D textures Lots of texture formats Wrapping modes Filter modes Mip-mapping Easy access from within shaders Shadow mapping Environment mapping 63
That s what s cool about working with computers. They don t argue, they remember everything, and they don t drink all your beer. Paul Leary