Framebuffer Objects. Emil Persson ATI Technologies, Inc.

Transcription

1 Framebuffer Objects Emil Persson ATI Technologies, Inc. Introduction Render-to-texture is one of the most important techniques in real-time rendering. In OpenGL this was traditionally handled with calls to glcopyteximage2d() and glcopytexsubimage2d(). The problem with this approach, however, is that window size limits the maximum possible resolution of the rendered texture, particularly with power-of-two textures. For example, the largest possible render target for an 800x600 window is only 512x512. Also, all render-to-texture operations must be completed in the beginning of the frame in order to not overwrite any content of the framebuffer. In response to all these limitations, PBuffers got utilized as a render-to-texture solution and quickly became the standard method for implementing render-to-texture in OpenGL. PBuffers permit off-screen rendering and allow resolutions independent of the main framebuffer, but require a copy to occur from the PBuffer data into a texture. To solve this problem WGL_ARB_render_texture was introduced, allowing applications to bind a PBuffer to a texture directly; avoiding the copy. This is where render-totexture in OpenGL has been for quite a while now. The solution has not been popular among developers, though, for several reasons, and many have looked to Direct3D s implementation of render-to-texture as a standard for how render to texture should really be implemented. One of the biggest complaints of PBuffers is their requirement for unique GL contexts. This means that the application must keep track of multiple sets of states, which is cumbersome and inconvenient. Additionally, context switching is an expensive operation. Another important issue with PBuffers is that each PBuffer has its own color, depth and stencil buffers and there is no way to share them. This is both expensive in terms of memory and limiting in terms of functionality. For example, it is not possible to perform depth-only rendering. Instead, a color buffer must always be present. Another problem is that PBuffers are windowing system dependent. PBuffers started off as an extension to GLX and later a WGL version was ratified. This means that applications wishing to run on more than one OS would have to use different code paths. Also, no GLX version of WGL_ARB_render_texture was ever ratified by the ARB (however, there is a GLX_ATI_render_texture). Other complaints are that initialization is cumbersome and too disconnected from regular textures, that it requires additional calls when binding a PBuffer as a texture, that there s no control over when the mipmaps are actually generated when automatic mipmap generation is enabled, and so on. Only the least desirable characteristic of Direct3D render targets is also present with PBuffers, namely the fact that buffers can get lost. So in short, OpenGL needed a better render-to-texture solution; enter the FramebufferObject extension.

2 The GL_EXT_framebuffer_object extension (FBO for short) is essentially built from scratch and does not carry with it the baggage of earlier render-to-texture solutions. FBO has all of the following attributes: Integrates directly with regular textures Requires no extra GL contexts Is windowing system independent Allows the application to control when mipmap generation occurs Allows rendering to a target without a colorbuffer Allows depth, stencil and color buffers to be shared, and No buffers ever get lost The basics Objects There are a number of important kinds of objects in GL_EXT_framebuffer_object. The central part of FBOs is the framebuffer object, which encapsulates all the state related to a framebuffer. A framebuffer in this context means a collection of logical buffers. A logical buffer is one of color, depth or stencil buffer. The logical buffers are created independently and are attached to framebuffer objects. A framebuffer object can have one or more color buffers, one depth buffer and one stencil buffer attached. A logical buffer can be attached to one or more framebuffers objects. Attach in this context means that the logical buffer gets connected to the object in a similar way as various bind operations (such as glbindtexture()) work, except that attaching does not create a new object if an unused value is passed to it. Each framebuffer object has a set of attachment points that logical buffers can be attached to. The attachment points are COLOR[n], DEPTH and STENCIL. The logical buffers can be attached to several framebuffer objects. Creating buffers To create a framebuffer object the glgenframebuffersext() command is called. Color buffers are created with the regular glgentextures() call and set up with the glteximage2d() function, except when you want to create a render target you typically pass NULL to the pixels parameter. It is valid however to provide an image and later overwrite it or parts of it by rendering to the texture. A typical FBO initialization code looks something like this:

3 GLuint fbo, color; // Create an FBO glgenframebuffersext(1, &fbo); // Create color texture glgentextures(1, &color); glbindtexture(gl_texture_2d, color); // Bind the FBO and attach color texture to it glbindframebufferext(gl_framebuffer_ext, fbo); GL_TEXTURE_2D, color, 0); To render to this color texture simply call: glbindframebufferext(gl_framebuffer_ext, fbo); And to return to rendering to the main framebuffer you would call: glbindframebufferext(gl_framebuffer_ext, 0); Now color can be used like any regular texture for rendering. Normally you would also like to use a depth buffer when rendering to a texture. Depth and stencil buffers are created with glgenrenderbuffersext() and set up with glrenderbufferstorageext(). The additional code for creating a depth buffer looks as follows: // Create depth renderbuffer glgenrenderbuffersext(1, &depth); glbindrenderbufferext(gl_renderbuffer_ext, depth); glrenderbufferstorageext(gl_renderbuffer_ext, GL_DEPTH_COMPONENT16, height); width, To attach it to the currently bound FBO you call: glframebufferrenderbufferext(gl_framebuffer_ext, GL_RENDERBUFFER_EXT, depth); GL_DEPTH_ATTACHMENT_EXT,

4 Again, to render to this texture you call glbindframebufferext() and then again with zero when you want to return to the main framebuffer. Changing render targets There are three different ways to switch between framebuffers. The first one is to use several different FBOs, one for each combination of logical buffers that you plan on using in you application. To change render targets you simply call glbindframebufferext() with the FBO containing the setup you wish to render to. The initialization code would look something like this: GLuint fbo[2], color[2]; // Create two FBOs glgenframebuffersext(2, fbo); // Create two color buffers glgentextures(2, color); glbindtexture(gl_texture_2d, color[0]); glbindtexture(gl_texture_2d, color[1]); // Attach the color buffers to the FBOs glbindframebufferext(gl_framebuffer_ext, fbo[0]); GL_TEXTURE_2D, color[0], 0); glbindframebufferext(gl_framebuffer_ext, fbo[1]); GL_TEXTURE_2D, color[1], 0); Then the rendering code would be as follows: // Render to color0 glbindframebufferext(gl_framebuffer_ext, fbo[0]); // Render to color1 glbindframebufferext(gl_framebuffer_ext, fbo[1]); // Return to rendering to main framebuffer glbindframebufferext(gl_framebuffer_ext, 0); Another way is to use a single FBO and alter the attachments. The setup could looks like this:

5 GLuint fbo, color[2]; // Create an FBO glgenframebuffersext(1, &fbo); // Create two color buffers glgentextures(2, color); glbindtexture(gl_texture_2d, color[0]); glbindtexture(gl_texture_2d, color[1]); And then the rendering code would be as follows: // Render to color0 glbindframebufferext(gl_framebuffer_ext, fbo[0]); GL_TEXTURE_2D, color[0], 0); // Render to color1 GL_TEXTURE_2D, color[1], 0); // Return to rendering to main framebuffer glbindframebufferext(gl_framebuffer_ext, 0); The third way is to use a single FBO, but instead of altering attachments you call gldrawbuffer() or gldrawbuffers() to change which color attachment(s) the rendering goes to. To set it up:

6 GLuint fbo, color[2]; // Create an FBO glgenframebuffersext(1, &fbo); // Create two color buffers glgentextures(2, color); glbindtexture(gl_texture_2d, color[0]); glbindtexture(gl_texture_2d, color[1]); // Attach the color buffers to the FBO glbindframebufferext(gl_framebuffer_ext, fbo); GL_TEXTURE_2D, color[0], 0); glframebuffertexture2dext(gl_framebuffer_ext, GL_COLOR_ATTACHMENT1_EXT, GL_TEXTURE_2D, color[1], 0); And for rendering: // Render to color0 glbindframebufferext(gl_framebuffer_ext, fbo); gldrawbuffer(gl_ COLOR_ATTACHMENT0_EXT); // Render to color1 gldrawbuffer(gl_ COLOR_ATTACHMENT1_EXT); // Return to rendering to main framebuffer glbindframebufferext(gl_framebuffer_ext, 0); Which method to use is mostly a personal preference or depends on what s most convenient in your case. The first method makes things easy in the rendering code, but the setup code is larger. The second method is the most general and most similar to the Direct3D model. The last method will likely have the least driver overhead (though this may change between drivers), but is also the most limited. You can only switch between color render targets this way. To change depth and stencil buffer you ll have to change the attachments. Render to cubemaps and volumes The most common render-to-texture operation is rendering to a regular 2D texture. Also fairly common is rendering to a cubemap. New in GL_EXT_framebuffer_object is rendering to a volume texture, something that could not be done with PBuffers.

7 Rendering into a cubemap is not that different from rendering to a 2D texture. You create an FBO as usual and a texture object. Then you size the six faces: // Create a cubemap color buffer glgentextures(1, &color); glbindtexture(gl_texture_cube_map, color); for (int face = 0; face < 6; face++){ glteximage2d(gl_texture_cube_map_positive_x + face, 0, GL_RGBA8, width, height, 0, } A cubemap is naturally rendered to face by face. To render to a particular face you attach that face to the FBO, for instance: GL_TEXTURE_CUBE_MAP_POSITIVE_X, color, 0); For volume rendering the procedure is about the same: // Create a volume color buffer glgentextures(1, &color); glbindtexture(gl_texture_3d, color); glteximage3d(gl_texture_3d, 0, GL_RGBA8, width, height, depth, 0, GL_UNSIGNED_BYTE, NULL); GL_RGBA, Rendering to a volume is not surprisingly done slice by slice. There s an extra parameter in the glframebuffertexture3dext() call that selects what slice in the volume to render to: glframebuffertexture3dext(gl_framebuffer_ext, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_3D, color, 0, zoffset); Render to depth Unlike with PBuffers it is possible to create an FBO with just a depth attachment and no color buffers. After rendering, the depth buffer can be used as texture directly (though the driver may have to do a copy on some hardware). This is useful for techniques such as shadow mapping. Creating a texture for depth rendering is similar to creating a color texture; we just need to choose a depth format:

8 // Create depth texture glgentextures(1, &depth); glbindtexture(gl_texture_2d, depth); glteximage2d(gl_texture_2d, 0, GL_DEPTH_COMPONENT16, width, height, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, NULL); This will be our depth buffer, but it is a texture and not a renderbuffer, so to attach it to the FBO we call glframebuffertexture2dext(), but hook it up to the depth attachment point: // Bind the FBO and attach depth texture to it glbindframebufferext(gl_framebuffer_ext, fbo); glframebuffertexture2dext(gl_framebuffer_ext, GL_DEPTH_ATTACHMENT_EXT, GL_TEXTURE_2D, depth, 0); By default the draw-buffer and read-buffer for an FBO is GL_COLOR_ATTACHMENT0_EXT, meaning that the driver will expect there to be a color buffer attached to render to. Since we don t have a color attachment the framebuffer will be considered incomplete. Consequently, we must inform the driver that we do not wish to render to the color buffer. We do this with a call to set the draw-buffer and readbuffer to GL_NONE: gldrawbuffer(gl_none); glreadbuffer(gl_none); Note that this is a per-fbo state, so if you re using separate FBOs for each render-to-texture setup, you can make these calls once at setup and forget it. If you re using a global FBO and altering attachments you need to restore the draw-buffer and read-buffer states for regular color rendering: gldrawbuffer(gl_color_attachment0_ext); glreadbuffer(gl_color_attachment0_ext); Mipmap generation For the best quality it is often desirable to have rendered textures mipmapped. Normally one would like the mipmap sublevels to be generated by down-sampling the base level. The GL_SGIS_generate_mipmap extension was usually used with PBuffers to generate the sublevel, and it s still valid to use it with FBOs. However, GL_EXT_framebuffer_object also adds a new glgeneratemipmapext() function for this purpose which allows more fine-grained control over when mipmap generation occurs. With this function the application can manually generate the mipmaps when it so desires, which is done like so:

9 glbindtexture(gl_texture_2d, color); glgeneratemipmapext(gl_texture_2d); This function may be applied to textures used as render targets or any regular texture; thus glgeneratemipmapext() can fully replace GL_SGIS_generate_mipmap. Multiple render targets Using multiple render targets with FBOs is a piece of cake. What we need to do is simply attach all the color buffers to the FBO s different color attachments points. Then the draw buffers need to be set. // Attach color buffers for MRT GL_TEXTURE_2D, color[0], 0); glframebuffertexture2dext(gl_framebuffer_ext, GL_COLOR_ATTACHMENT1_EXT, GL_TEXTURE_2D, color[1], 0); glframebuffertexture2dext(gl_framebuffer_ext, GL_COLOR_ATTACHMENT2_EXT, GL_TEXTURE_2D, color[2], 0); glframebuffertexture2dext(gl_framebuffer_ext, GL_COLOR_ATTACHMENT3_EXT, GL_TEXTURE_2D, color[3], 0); // Setup draw buffers GLuint drawbuffers[4] = { GL_COLOR_ATTACHMENT0_EXT, GL_COLOR_ATTACHMENT1_EXT, GL_COLOR_ATTACHMENT2_EXT, GL_COLOR_ATTACHMENT3_EXT, }; gldrawbuffers(4, drawbuffers); Note that even though GL_EXT_framebuffer_object provides an interface for MRT it may not be supported on all implementations. You should check if GL_ARB_draw_buffers or OpenGL 2.0 is present, and you must call glgetintegerv() with GL_MAX_DRAW_BUFFERS to see how many drawbuffers the implementation supports. Some hardware and/or drivers may only support a single color buffer. Additional restrictions apply, too, such as all draw-buffers must have the same size and internal format. The latter restriction is expected to be alleviated in an additional extension. Porting from PBuffers To context-hell and back When porting existing code using PBuffers to FBOs it s easy to forget the fundamental differences in behavior between the two. This will more than likely cause a good deal of headaches in anything but trivial applications. Chances are that the change also exposes bugs you ve [un-]luckily been dodging all the time.

10 As in real life, removing barriers is certainly a good thing, but undoubtedly unwanted elements will flow over the borders too, in both directions. That s certainly the case when breaking the walls to the PBuffers protected little world inside their own GL-contexts. Undesirable GL-states will surely flow in and out of your render-to-texture code. Whether intentional or not, applications built around PBuffers are likely to set global states and expecting them to live the entire frame, or even the entire lifespan of the application. An application may for instance set the projection matrix in the beginning of each frame and be happy with that. When rendering to the PBuffer it has its own projection matrix, so if it changed as part of the render-to-texture operation it only affected the current PBuffer, and the main context had its matrix unaffected. When changing this to FBOs it obviously causes problems. Fixing the code to restore the projection matrix is generally not a problem, but finding out that this is the reason why the rendering totally broke down can be quite tricky. Similarly, when rendering to the texture the application could easily set a bunch of states, and never care about restoring any of them when it s done since they only affect that particular context. When changing to FBOs those states will of course survive over to the rendering passes that follow. One of the first issues you ll likely bump into when bringing up your first FBO in your application is that things look stretched, are packed in the corner of the render target or nothing shows up at all, in particular with very large or small textures. With PBuffers having their own context they also have their own viewport state. So when switching to a PBuffer, the viewport would automatically change that in the PBuffer s context, which defaults to the exact size of the PBuffer. Not so with FBOs though since they don t have a context of their own. Instead, the viewport is still that of the main framebuffer so unless the render target is the exact size of the main framebuffer you have to call glviewport() to set the viewport to the size of the render target, and then back to the size of the main framebuffer again when you re done. There are of course loads of states that could cause problems, but some common ones to look for are: The projection and modelview matrices Blending mode/op/factors Depth test Color and depth masks Vertex array enables, and Cull face Other things like clip planes and scissor rectangles can also cause problems. Good practices Check the framebuffer status The first release of this extension contains no mechanism to enumerate supported formats and framebuffer configurations. This is likely to be solved in a future extension, but for now it s up to the application to check whether a particular framebuffer configuration is complete, that is, valid for rendering to, and if needed try other formats until it succeeds or there are no suitable formats left to try. This can be done with the glcheckframebufferstatusext() function. It is highly

11 recommended to call this function before rendering after the FBO has been set up. It is also very helpful for debugging to check the error codes returned for FBO configuration problems. For instance in the render-to-depth case discussed above, had the draw-buffer and read-buffer not been set to GL_NONE, glcheckframebufferstatusext() would return the GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT error, hinting where the problem lies. The possible return values are as follows: GL_FRAMEBUFFER_COMPLETE_EXT The framebuffer is complete and valid for rendering. GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT One or more attachment points are not framebuffer attachment complete. This could mean there s no texture attached or the format isn t renderable. For color textures this means the base format must be RGB or RGBA and for depth textures it must be a DEPTH_COMPONENT format. Other causes of this error are that the width or height is zero or the z-offset is out of range in case of render to volume. GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT There are no attachments. GL_FRAMEBUFFER_INCOMPLETE_DUPLICATE_ATTACHMENT_EXT An object has been attached to more than one attachment point. GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT Attachments are of different size. All attachments must have the same width and height. GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT The color attachments have different format. All color attachments must have the same format. GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT An attachment point referenced by gldrawbuffers() doesn t have an attachment. GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT The attachment point referenced by glreadbuffers() doesn t have an attachment. GL_FRAMEBUFFER_UNSUPPORTED_EXT This particular FBO configuration is not supported by the implementation. All errors are universal and will fail on all implementations, except GL_FRAMEBUFFER_UNSUPPORTED_EXT which is implementation specific. This error can be caused by limitations in the underlying hardware or other restrictions of the implementation. Finally, glcheckframebufferstatusext() may also return zero if the function itself generates an error.

12 Note that it s legitimate to straddle through incomplete configuration states while setting up the FBO. No attention has to be taken to attach and detach in a particular order to always have complete configurations on the way from one setup to another. The framebuffer completeness will only be validated on draw calls and when glcheckframebufferstatusext() is called. Calling glcheckframebufferstatusext()does not generate an error when the framebuffer is incomplete, but returns the status so that the application can take proper action. Only if a draw call is issued when the framebuffer is in an incomplete state will an error (GL_INVALID_FRAMEBUFFER_OPERATION_EXT) be issued. Use properly specified textures Textures intended for rendering to should be properly setup at an early stage. If you plan on using mipmapping on a render target it s best to allocate all mipmap levels at once at creation time, rather than relying on glgeneratemipmapext() to create them for you. Otherwise the driver may have to reallocate the texture and possibly copy over the base level contents to a new location. Also be sure to correctly set the desired minify filter of all textures to be used as render targets before attaching to a framebuffer object. Failing to do so may have the driver allocating mipmaps for render targets that don t need them. Don t mix and match For best performance it s recommended that you keep your regular textures and textures used as a render targets separate. Don t modify your render targets by issuing calls such as gltex{sub}image2d or glcopytex{sub}image2d. Doing so may incur heavy penalties for reallocation, memory copying and data conversion. This is because render targets and textures have different requirements from the hardware s point of view and may not support the same set of formats and memory layouts. If you need to update a subsection of a render target, use rendering commands such as gldrawpixels() instead. Conclusion GL_EXT_framebuffer_object is an easy to use extension that solves all the major problems of PBuffers. It is intended to be a full replacement solution for off-screen rendering. However, in order to shorten the time-to-market of this extension not all features were included. Things not included are support for multisampling, accumulation buffers and a way to communicate what formats and configurations the implementation supports. Other features such as render-to-vertex-array are also being considered. Additional extensions targeting some or all of these issues are expected to follow. Once these issues are settled and enough feedback from developers and implementers has been assembled it is expected that this extension, possibly including additional extensions built on top of it, will receive the ARB s blessing. For now this should give a good starting point for writing good render-to-texture applications with a minimum amount of hassle.