High-Degree Temporal Antialiasing

Transcription

1 High-Degree Temporal Antialiasing Frank Dachille IX and Arie Kaufman Center for Visual Computing (CVC) and Department of Computer Science State University of New York at Stony Brook Stony Brook, NY , USA Abstract We propose the use of high-degree resampling filters for improved temporal antialiasing, or as the result is often called, motion blur. Without temporal antialiasing, strange effects can occur within an animation, for example, wheels can appear to spin backwards at a certain speed. In a typical effort to overcome this, the camera shutter is left open over some period of time during the frame, leading to temporal box filtering. Even with a temporal box filter, aliasing can still occur. We show how a high-degree resampling filter, such as the Gaussian or cubic-spline lead to superior results in all cases, without the need for constant scrutiny and hand tweaking on the part of the animator. 1. Introduction We have all seen at one time or another the visual result of temporal aliasing. Perhaps viewing an old western movie in which the wagon wheels began to spin backwards at some speed or shaking your finger in front of a television or computer monitor to generate a strobing effect, you have experienced it. The root of this aliasing is the temporal comb filter that samples only at discrete times over the motion path of an object. In the case of wagon wheels, the shutter speeds used by early filmmakers effectively sampled only at discrete times. In the case of the computer monitor, the screen refresh rate provides significant back lighting for your finger at only a few discrete times, yielding a strobing effect. In computer graphics, we have the ability to open the virtual camera shutter any amount at any time during an animation (see [5] for explanations and lucid space-time visualizations of various real and fanciful shutter mechanisms). Due to the limited frame rate of conventional visual output devices, we can only provide information at a few discrete times (in addition to the limited image plane resolution). If no temporal antialiasing is used, a strobing effect is generated during movement of an object across the image plane. Any finite frame rate, no matter how high, has the potential for aliasing depending on the relative speed or frequency of object movement. The conventional technique to combat temporal aliasing in computer graphics animation is to open the shutter for some finite period of time around the frame time, generally for the length of the inter-frame time centered around the current frame [2, 9, 13]. However, armed with knowledge of sampling theory, we can do better than this. Early investigations into simulating motion blur [9, 13] suggested blurring moving objects by sampling them at various times and weighting the contribution of each sample. They suggested equally weighted samples over a specified time period, although they mentioned the possibility of other weightings. Care must be taken to use the proper number of samples to reach a continuous tone blur; too few samples results in distracting strobing. Hardware has been developed which can accommodate this form of motion blur by averaging multiplesamples for each frame using an accumulation buffer [7, 14]. We examine improved methods of weighting samples over time which require unequal weightings and a greater number of samples. Distribution ray tracing [2] improved the quality of images by trading aliasing (strobing seen by averaging just a few points in time) for noise by stratifying samples [4] across multiple dimensions (space, time, reflection angle, glossy refraction angle, etc) separately for each sample of each pixel. Commercial implementations of RenderMan use the principle of distribution ray tracing, allowing various filters (e.g., Bartlett, Gaussian, sinc) to be used to stratify samples across the image plane. We use the general idea of distribution ray tracing to stratify samples across domains, but instead use improved filters for weighting contributions in the time domain. Another approach for effective antialiasing (including temporal) is to pre-filter in object space prior to rendering. This technique is difficult to use in practice because it requires computing the paths of objects through space prior to

2 rendering. It has been used to blur both polygonal [15] and volumetric objects [3]. Object-space temporal antialiasing has also been used as a post-process for polygons [10] and raster objects [11]. However, these techniques are limited by being object specific, path specific, and not general in nature. The remainder of the paper is organized as follows. We investigave filtering theory and its relation to temporal antialiasing in Section 2. We discuss our method for filtering a static image in Section 3. We progress to filtering dynamic images in Section 4. We present our results and discussion in Section 5. Finally, we draw our conclusion in Section ing We begin by first examining sampling theory, the basis of antialiasing. In order to represent a continuous object using only discrete symbols, we need to first bandlimit the object, then sample it at discrete locations. In other words, we need to clamp the original signal to allow only low frequency content which can be faithfully represented at the available output rate. A signal f(t) can be filtered by convolving it with a low pass filter h(t) to remove all the high frequency content f 0 (t) =f(t)λh(t): The sinc filter (sinc(x) = sin(ßx)=ßx) is the ideal bandlimit filter, but it is impractical to realize due to its infinite support. The box filter, which equally weights all the samples over a specified period of time, is most often used in practice. However, we know from sampling theory that box filtering of any signal has a poor frequency response; box filtering in the spatial domain causes ringing in the frequency domain. In our application, box filtering in the temporal domain changes (or aliases) the apparent speed of motion. To avoid aliasing of moving objects, we need to apply a better filter. We can improve on the box filter by using a Bartlett (or triangle) filter. We can do even better with a Gaussian filter, a cubic B-spline filter, a Catmull-Rom cubic spline, or the generalized spline filters of Mitchell and Netravali [12]. The set of filters can be represented by spline curves of increasing degree. Figure 1 shows a comparison of several resampling filters from low to high-degree. 3. Static Image ing We begin an examination of temporal antialiasing by first rendering a still image. We have chosen a star image [16] that contains a wide range of frequencies, many of which can not be displayed at typical screen resolutions (see Figure 2). Thus, representing this image is a good test of any Quality Time Gaussian Triangle Box Comb Figure 1. A comparison of several resampling filters of increasing quality. sampling filter. For spatial antialiasing, we used a normalized Gaussian filter 2 2 =ff h(t) = e t p 2ßff 2 with ff = 1.0 windowed to a total width of six pixels. The value of ff was hand-selected to balance the opposing goals of minimal blurring and maximal crispness. Too narrow a filter approximates point sampling resulting in spurious and distracting aliasing. Too wide a filter over-blurs the image, wasting pixels and losing detail. 4. Dynamic Image ing If we now animate the star image from left to right, we have a very demanding test of any spatio-temporal filter. A simple demonstration shows how even the real world includes aliasing. You can induce visually distracting moiré effects by simply shaking Figure 2 from left to right or just by reading the caption. Depending on the speed and direction of movement, various patterns emerge. In computer graphics, we often wish to avoid distracting artifacts, even if they may be physically based. Our method to eliminate these is to filter out all those frequencies which would cause aliasing in the final image. If during the animation we open the shutter for an instantaneous moment of time as it crosses the middle we have simulated the comb temporal filter (the lowest graph in Figure 1). A single frame from a comb filtered animation appears crisp, but aliased when viewed in sequence (e.g., stop-action claymation). If we instead quickly open the shutter for a short period of time and quickly shut it, we have achieved a box temporal filter. Since we have the ability to precisely control the aperture of the shutter (the

3 Figure 2. The original star image with spatial Gaussian filtering. Figure 4. An enlarged portion of Figure 3b showing a reversal in the black/white pattern at certain frequencies (harmonics). amount of contribution over time), we can slowly open and slowly close the shutter over time resulting in a Bartlett temporal filter. Finally, we can select an even better filter such as a Gaussian or Catmull-Rom cubic spline to weight our contribution over time. Each progressively better filter requires a larger filter support and greater processing time. If we were sampling a 1D function, then we would use a 1D filter function. But since we in this particular case are sampling a 2D function over time, we are actually sampling a 3D function. In this way it makes sense to use an integrated filter, one that samples all three dimensions simultaneously. One approach is to distribute the samples more or less equally and modify the filter weight for each sample. A better approach is to use importance sampling by equally weighting the samples, but concentrating the number of samples to approximate the shape of the filter function. Therefore, to sample with a Gaussian filter, we can generate the samples using a Gaussian distributed sample generator instead of a uniformly distributed sample generator. Thus, we select three Gaussian distributed random numbers to sample the three dimensions of our space (x, y, and t) and weight the samples evenly. 5. Results and Discussion Figure 3b demonstrates box filtering the star image as it moves from left to right. Note the resulting aliasing on the top and bottom, perpendicular to the direction of travel. By increasing the filter quality (recall the prior discussion on resampling filters) we can reduce the aliasing to undetectable levels. If we instead filter using a Bartlett filter (see Figure 3c), the image is only slightly aliased. Ringing occurs at some frequencies, causing subtle rings to appear. For the highest quality and the longest running time we can filter with a Gaussian filter (see Figure 3d). There is no detectable aliasing, particularly when viewed as an animation. The most surprising result of the above exercise is found in a small portion of Figure 3b which has been expanded into Figure 4. It shows that a certain band of spatial frequencies severely alias when translated at a certain speed and filtered with the box filter. This pattern occurs when the black/white pattern has a period of two units (i.e., the black and white strips are each one unit wide) and the box filter spans exactly three units. If a box filter is centered on a white strip, then it spans exactly two black strips and only one white strip, incorrectly leading to an average closer to black than to white. This same effect can be observed in a more natural setting. We have generated an animation visualizing a white picket fence viewed out of an accelerating car s window (see Figure 5). We used the freely available RenderMancompliant Blue Moon Rendering Tools (BMRT) [6] for rendering. In the first frame, the viewer is stationary; in subsequent frames, the car s velocity increases linearly The strange effect occurs in Figure 5d. Not only should the

4 fence be motion blurred, but it even appears in the wrong location. A close comparison of Figure 5a and Figure 5d shows that the fence posts appear in the originally empty space between the fence posts. There is usually no need to progress to an even higher quality filter than the Gaussian. As demonstrated in the Gaussian blurred star, all frequencies of interest cause no noticeable aliasing. The star image is a helpful test image since it contains all frequencies of interest for antialiasing in approximately all directions. If the star image is antialiased, then any image containing the represented frequencies will be antialiasing if filtered the same way. We tested this on our picket fence animation by rectifying the aliasing of Figure 5d. Because the RenderMan standard does not currently support high-degree temporal filtering, we had to perform the filtering ourselves. We rendered the animation using the available box filtered motion blur at 20 times the final frame rate. We then resampled the oversampled frames into the final animation using the various filters we have discussed. Figure 5f is a frame from the temporal Gaussian filtered animation. It corresponds exactly to the box filtered Figure 5d. Efficiency can be improved by using a quasi-monte Carlo (QMC) method, in which pseudo-random samples are replaced by low-discrepancy or quasi-randomsequences (e.g., [8]). By distributing the available finite samples more evenly over the integrand, we can achieve faster convergence to the expected value of the pixel, thus lowering the total number of samples required to generate the complete image. Of course, QMC methods can be applied to the integration of almost any function (e.g., distribution ray tracing [2]). 6. Conclusion We have presented a temporal antialiasing method which demonstrably improves upon conventional motion blur methods by utilizing a high-degree resampling filter. Using this temporal antialiasing filter avoids the labor-intensive scrutiny and hand tweaking required to eliminate aliasing using the usual box filter for antialiasing. While the occurance of aliasing under the temporal box filter is admittedly infrequent, use of a higher-degree resampling filter almost completely eliminates the problem under all circumstances. Perhaps even higher quality can be achieve at lower cost by analytically computing the intersection of a space-time hypercone with the moving objects in a scene. This could be accomplished using a multi-dimensionalexpansion of cone tracing [1]. We would advise developers of computer animation systems to implement higher-degree temporal filtering, for it improves the general quality of animations. However, the extra time required to render with this filter can be large, thus it is best reserved as an option for high-end, production quality output, not for general use. Acknowledgments This work has been supported by ONR grant N The authors wish to thank Justine Dachille for her patience during this project. References [1] J. Amanatides. Ray tracing with cones. Computer Graphics (SIGGRAPH 84 Proceedings), 18(3): , July [2] R. L. Cook. Stochastic sampling in computer graphics. ACM Transactions on Graphics, 5(1):51 72, Jan [3] F. Dachille and A. Kaufman. Incremental triangle voxelization. To appear in Graphics Interface 2000, [4] M. A. Z. Dippe and E. H. Wold. Antialiasing through stochastic sampling. Computer Graphics, 19(3):69 78, July [5] A. Glassner. An open and shut case, Andrew Glassner s Notebook. Computer Graphics and Applications, 19(3), May [6] L. Gritz. Blue Moon Rendering Tools. Web page. [7] P. Haeberli and K. Akeley. The accumulation buffer: Hardware support for high-quality rendering. Computer Graphics (SIGGRAPH 90 Proceedings), 24(4): , Aug [8] A. Keller. Instant radiosity. In SIGGRAPH 97 Conference Proceedings, Annual Conference Series, pages 49 56, Aug [9] J. Korein and N. Badler. Temporal anti-aliasing in computer generated animation. Computer Graphics, 17(3): , July [10] N. L. Max. Polygon-based post-process motion blur. The Visual Computer, 6(6): , Dec [11] N. L. Max and D. M. Lerner. A two-and-a-half-d motionblur algorithm. Computer Graphics (SIGGRAPH 85 Proceedings), 19(3):85 93, July [12] D. P. Mitchell and A. N. Netravali. Reconstruction filters in computer graphics. In Computer Graphics (Proc. SIG- GRAPH 88), volume 22(4), pages , July [13] M. Potmesil and I. Chakravarty. Modelling motion blur in computer-generated images. Computer Graphics, 17(3): , July [14] G. Tarolli. Real-time cinematic effects on the PC: the 3Dfx T-buffer. unpublished presentation at the 1999 SIGGRAPH / Eurographics Workshop on Graphics Hardware, Aug [15] M. M. Wloka and R. C. Zeleznik. Interactive real-time motion blur. The Visual Computer, 12(6): , [16] G. Wolberg. Digital Image Warping. IEEE Computer Society Press, Los Vaqueros Circle, Los Alamitos, CA, 1990.

5 Figure 3. Four versions of the star image sampled during horizontal translation with various temporal filters. As the center of each star passed through the center of the image, the virtual shutter was opened according to temporal (a) comb, (b) box, (c) Bartlett or triangle, and (d) Gaussian filter. Spatial Gaussian filtering was applied in all cases.

6 Figure 5. A sequence of temporal box filtered animation frames showing a white picket fence viewed out of the window of an accelerating car. The speed is (a) 0, (b) 1, (c) 2, (d) 3, and (e) 5 fence slats per frame. Note the aliasing in frames (d) and (e) which are third and fifth harmonics of the fence slat frequency. Furthermore, a careful examination of (b) and (d) shows the pattern reversal predicted by Figure 4. A Gaussian filter was applied instead of the box filter to generate (f) which has a speed of 3 fence slats per frame and corresponds directly with (d). The Gaussian filter properly blurs the passing scenery and corrects the aliasing of the conventional box filter.