High Dynamic Range Image Texture Mapping based on VRML

High Dynamic Range Image Texture Mapping based on VRML Sung-Ye Kim and Byoung-Tae Choi 3D Graphics Research Team, Virtual Reality Research and Development Department, Computer Software Research Laboratory, Electronics and Telecommunications Research Institute, 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350, Korea {inside, btchoi@etri.re.kr http://insideblue.hihome.com, http://www.etri.re.kr Abstract. In this paper, we describe a technique for representing and displaying high dynamic range image (HDRI) as a texture map in VRML structure. To do this, we designed and implemented extended texture nodes for supporting HDRI in VRML and developed HDRI-based mapping tool that can be used for authoring objects with HDR image as a texture data. Even though the authoring result of our tool is not identical with standard VRML format exactly, it is good enough to be used in virtual world. To verify this, we developed an extended VRML browser that supports a parsing and viewing of our new texture nodes related to HDRI and allows the exposure to be adjusted continuously and arbitrarily at the time of navigation. 1 Introduction Recently, since a number of image-based techniques have begun to make appearance, it is possible to render realistic images with pretty low costs through the combination of computer vision and computer graphics technologies in modeling and rendering fields. However, 2D images as input data for an image-based technique intrinsically have limitations of representing all information being in the real world. Since the late of 1990 s, information of real or natural lights has been used as source data for image-based techniques. Especially, such fields are called by an image-based lighting, and many researches[7][8] have been accomplished until now. High Dynamic Range Image is the image data type can represent real light in the world. The dynamic range of a scene is the ratio between its the brightest and darkest parts, and a HDRI is an image that has a greater dynamic range than LDR (Low Dynamic Range) that can be shown on a standard display device or be captured with a standard camera with a single exposure. Properties of a HDRI are as follows. The first, the pixel values of HDRI are directly proportional to the amount of light in the world corresponding to that pixel. Therefore it is possible to be represented the ratio of a bright and dark region precisely in HDRI. The second, various realistic optical effects such as a HDR fresnel, auto exposure, glow and vignette can be expressed. Because of these characteristics,

2 Sung-Ye Kim et al. there have been a lot of researches related to application using HDRI. At present, various researches related to HDRI show the future course of realistic rendering in computer graphics. Tools[1] and plug-ins[3][4][5] for supporting HDRI have been produced by some researchers and developers. HDRShop[1] made by Paul Debevec is one of the most famous tools. This provides various HDR image processes and manipulations of HDRI such as loading, saving, editing, panoramic transformation, camera curve calibration, diffuse/specular convolution and assembling of HDRI using a series of images taken under different exposure. The LightGen[6] is an HDRShop compatible executable by Jonathan Cohen for generating a set of directional sources to approximate a light probe image. HDRI Bitmap I/O plug-in for 3D Studio Max and its tutorial are also available on website[3]. However, these only provide I/O operations for supporting HDRI in the existing rendering and modeling tools. 2 Texture Image nodes using HDRI In VRML, Appearance node is used to describe characteristic of an object, and each node of VRML consists of some fields. One of fields of Appearance node is a texture field that specifies the type of image for texture mapping. In this section, we ll explain the design and implementation of our new texture image nodes for supporting HDRI as a texture image in VRML-like environment. 2.1 Representation of new Texture Image nodes Table 1 shows the types of texture that can be used as the texture field of Appearance node. VRML defines three texture types like the ImageTexture, the MovieTexture and the PixelTexture. To expand an image type of texture field, we added new texture nodes by the name of the HDRImageTexture and the HDRICubeMapTexture nodes. The HDRImageTexture node was added as a new node with the purpose to use HDRI as a texture image for texturing of objects in VRML scene. By using HDRI as a texture image, we can get some advantages of HDRI in itself. For example, we can adjust the exposure of an environment in virtual world freely. Table 2 shows the specification of the HDRImageTexture node. It seems like an ImageTexture node except an exposure field of SFFloat type. We can also create the HDRICubeMapTexture node based on above node of Table 2. Table 3 shows the specification of this node. Six HDRI maps are needed to do cube mapping. 2.2 Implementation of our Texture Image nodes When we implement the HDRImageTexture node for our application, it has actually such the structure as Table 4. For the structure of a floating-point image data for a single HDRImageTexture, there is a pointer variable for image of uncompressed 32-bits floating-point value type, like floatrgbimage* floatimage. The structure of class of HDRICubeMapTexture node is similar to that of above

Lecture Notes in Computer Science 3 Table 1. Specification of Appearance node in VRML and image types of texture field in it. We added new texture nodes by the name of the HDRImageTexture and the HDRICubeMapTexture nodes. Appearance { ExposedField SFNode material NULL ExposedField SFNode texture NULL ExposedField SFNode texturetransform NULL texture field: - ImageTexture - MovieTexture - PixelTexture - HDRImageTexture (new) - HDRICubeMapTexture (new) Table 2. HDRImageTexture node. This seems like an ImageTexture node except an exposure field. The difference between two nodes in url field is the fact that url field of our node will have HDRI file. HDRImageTexture { ExposedField MFString url [] field SFFloat exposure 1.0 field SFBool repeats TRUE field SFBool repeatt TRUE Table 3. HDRICubeMapTexture node. This node have six url fields to represent a cube map and these urls are all HDRIs. HDRICubeMapTexture { ExposedField MFString urlright [] ExposedField MFString urlleft [] ExposedField MFString urltop [] ExposedField MFString urlbottom [] ExposedField MFString urlfront [] ExposedField MFString urlback [] field SFFloat exposure 1.0

4 Sung-Ye Kim et al. node. Because HDRICubeMapTexture node should have six maps, a vector for tvfloatrgbimage, std::<vector tvfloatrgbimage*> floatimagevector, will be needed to store them. Fig. 1 shows the execution result of a scene using two texture nodes. Table 4. Class structure for HDRImageTexture node. tvtexture is the name of a texture node that is offered in TOV library 1.0. We used TOV library to implement common VRML browser. class HDRImageTexture : public tvtexture { // for VRML 2.0 fields private: tvstringvector* url; tvfloat exposure; // member data private: tvfloatrgbimage* floatimage; tvbool imagechanged; Fig. 1. Screen shot of a mapping using HDRImageTexture and HDRICubeMapTexture nodes. A sphere was mapped by cube map using HDRICubeMapTexture and a teapot was mapped by sphere map using HDRImageTexture node. Table 5 shows the source of the scene using the HDRImageTexture and HDRICubeMapTexture nodes and the source file can be exported by our application. We also added texgen node for generating texture coordinate automatically. In Table 5, SPHERE MAP of texgen node specifies the mode for a

Lecture Notes in Computer Science 5 Table 5. A scene source using HDRImageTexture and HDRICubeMapTexture nodes. Fig. 1 shows the execution result of this source. In second line, TOV toolkit V1.0 is the name of a rendering library for VRML scene and that was used in our application. #VRML V2.0 utf8 #TOV Toolkit V1.0 Transform { translation -3 0 0 children [ Shape { appearance Appearance { texture HDRICubeMapTexture { urlright cmright.tif urlleft cmleft.tif urltop cmtop.tif urlbottom cmbottom.tif urlfront cmfront.tif urlback cmback.tif exposure 1.0 texgen TexGen { mode CUBE MAP geometry Sphere { ] Transform { translation 3 0 0 children [ Shape { appearance Appearance { texture HDRImageTexture { url insidehdri0.tif exposure 1.5 texgen TexGen { mode SPHERE MAP geometry Teapot { ]

6 Sung-Ye Kim et al. spherical texture mapping and CUBE MAP specifies that for a cube mapping. As well known, cube mapping is view-independent. We also added the Utah Teapot node as geometric primitive. 3 HDRI-based application for supporting HDRI texture mapping We implemented a HDRI-based application for supporting the texture mapping using HDRI. This section is composed of two parts. Those are the authoring tool part for HDRI texture mapping with HDRImageTexture or HDRICubeMapTexture nodes and the extended VRML browser for supporting our new texture nodes. 3.1 insidehdr: HDRI-based Authoring Tool Fig. 2 shows the entire picture of our tool, insidehdr. The main processing of it is composed of three parts. See operations from the PROCESSING (1) to (3). Fig. 2. Left picture shows the flow of the application from input to output. The processing part is composed of three parts. Right picture shows the snapshot at each step and the flow of operation. The essential function of insidehdr is a texture mapping using HDRI and also we proposed a cube mapping using HDRI except basic mapping types were specified in VRML. Even though these aren t new research entirely, we applied these functions to VRML environment by creating new texture nodes. In relation to HDRI texture mapping, there was a research[8] by J.Cohen. His paper described the technique for representing and displaying high dynamic range texture maps (HDRTMs) using current graphics hardware by decomposing into sets of 8-bit textures HDRI. However, we don t use a decomposing method. We just create VRML-like nodes by applying tone mapping to HDRI and they are

Lecture Notes in Computer Science 7 composed a scene graph. The insidehdr also provides some methods to manipulate HDRI such as loading, saving, diffuse/specular convolution, panoramic transformation, extract of light information and interpolation between specular radiance maps. These are same operations with those of HDRShop[1] by P. Debevec and that of LightGen[6] by J. Cohen. In our tool, we represented the result of extracting a set of directional lights with user-friendly user interface. See Fig. 3. Fig. 3. Generation a set of 3D lights from HDRI. We can display the direction of lights with 3D arrows and 2D points and save the directions and colors of lights in text file. Above picture shows 3D representation of 5 lights extracted from HDRI on right side. Below code is the function for applying new texture nodes to an object. We also provide common operations using ImageTexture and PixelTexture nodes of VRML. But such functions were omitted in below code. void applytexture() if (no texture data) if (CUBE_MAP) // 1.Create HDRI cube map node, HDRICubeMapTexture // 2.Define the name of the node to save as a file // 3.Save the HDRI data to HDRICubeMapTexture else // 1.Create HDRI texture map node, HDRImageTexture // 2.Define the name of the node to save as a file // 3.Save the HDRI data to HDRImageTexture else // already have texture data if (CUBE_MAP) if (no change of mapping data type as HDRI->HDRI) else // LDRI->HDRI else if (no change of mapping data type as HDRI->HDRI) else // LDRI->HDRI

8 Sung-Ye Kim et al. Fig. 4 shows the authoring of a car located in virtual showroom and the appearance of our tool. In the Fig. 4, the car model was mapped with HDRImageTexture of virtual show room. HDRI of the room was made with MAX5.0 manually. Fig. 4. This figure shows our HDRI-based authoring tool, insidehdr. This tool has various operations for manipulation of HDRI, changing of material of an object, and rendering using HDR texture image. This presents the authoring process of a car in virtual environment. 3.2 Extension of VRML browser for supporting new texture nodes To use HDRI as a texture image, common VRML browser should have the routines for processing like parsing and displaying of the nodes related to HDRI. That is to say, some routines for parsing of our new nodes and displaying of floating-point typed image are needed in browser. Generally operation for display of floating-point typed image data is called as a tone mapping. By operating the routine, it is possible to use HDRI as texture image in our extended VRML browser. It may be very simple. Implementation of the extended browser is based on TOV library 1.0. TOV offers functions for parsing nodes in VRML2.0 specification and displays them. So, we can create new nodes by inheritance of tvtexture node that is a texture node in TOV 1.0 library.

4 Results and Future Works Lecture Notes in Computer Science 9 We have demonstrated that high dynamic range images can be used as texture data in an extended VRML structure based on the Internet. The original goal of this paper is to develop the technique making possible use HDRI as a texture map in virtual space. So we designed new texture nodes related to HDRI in texture field of Appearance node in VRML and then developed HDRI-based mapping tool, insidehdr. By using the tool, we could create the object mapped with HDRI and save its authoring result as VRML-like formatted file. That is not identical with standard VRML but extended format. The viewer of our extended VRML browser based on OpenGL is used as a main window in insidehdr tool. Because it supports most operations as a web browser, as you can see in Fig. 6, we can also add JavaScript operations such as buttons for adjusting exposure and rotation of a model in our exported result. The insidehdr provides three main functions. The first, we can manipulate HDRI. For example, we can get different formatted HDRI, 3 dimensional light information and diffuse or specular radiance map from angular map Light Probe image and so on. The second, we can create VRML scene supporting HDRI as an image type of texture node. The final exported VRML file can be loaded and navigated by our extended browser. And then it is possible to adjust an exposure of that virtual environment in real time. In near future, because most of common browsers don t support HDRI type, we will also consider embedding of our work in the common VRML browser. Fig. 5. (Left) A series of LDR cube maps images made under different exposure. We make HDRI by assembling them. (Right) Angular maps formatted HDRI created with left cube map HDRIs. (See Fig. 6 for the application of the usage of these HDRIs) References 1. http://www.debevec.org/hdrshop/main-pages/tutorials.html 2. http://www.debevec.org/probes

10 Sung-Ye Kim et al. Fig. 6. Virtual showroom. In this room, a car was mapped with HDRI texture map of Fig. 5. 3. http://www.splutterfish.com/gabry hdri tut tmp 4. http://www.lightwave3d.com/tutorials/fullprecision/index.html 5. http://www.ocf.berkeley.edu/ eliza/cs184 6. http://www.ict.usc.edu/ jcohen/lightgen/lightgen.html 7. Paul Debevec: Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography. In Proceeding of SIGGRAPH 98, (1998) 189 198 8. Jonathan Cohen, Chris Tchou, Tim Hawkins, and Paul Debevec: Real-time High Dynamic Range Texture Mapping. In Eurographics Rendering Workshop, London, (2001) 9. Greg J. Ward: Real Pixels. Graphics Gems II, Edited by James Arvo, San Diego: Academic Press (1991) 80 83