Real-Time Rain Rendering in City Environments

Transcription

1 Artist-Directable Real-Time Rain Rendering in City Environments Natalya Tatarchuk 3D Application Research Group ATI Research

2 Overview Rain rendering: introduction Related work Rendering rain precipitation GPU Water effects Results discussion

3 Introduction

4 The Challenge of Rendering Rain Rain is hard Creating convincing impression of rain in rich natural environments is a difficult task It is a complex phenomenon Vast diversity of rain components Huge amount of small details However: rain rendering greatly enhances outdoor scenes Many applications in games and motion pictures Challenging to film and even more so to render at interactive rates

5 Desired Goals Combine the individual components in an interactive environment Therefore each algorithm should be efficient and flexible Yet must produce high quality physicallyplausible visual results Easy adoption into production pipelines High degree of artistic control

6 Related Work

7 Related Work: Topics Modeling and rendering rain precipitation Rainfall Raindrops Rain splashes Water simulation for puddle rendering Fluid dynamics Approximation effects Atmospheric effects Lightning illumination Fog and multiple scattering effects Rain droplets simulation and rendering Reflections rendering

8 Related Work in Rain Precipitation Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006 Rain effects have been examined in the context of atmospheric sciences and computer vision Much research on the rain phenomena in atmospheric sciences ([Mason75] and [Wang75]) Constant brightness rain strokes generated in [Starik03] for simulation of rain in videos Snow and rain precipitation in [Wang04] Modeled with interpolated hand-drawn textures With constant brightness Mapped on a camera-aligned double-cone At the moment, only a few approaches exist for creating realistic rainfall rendering [GN06] Challenge: Dynamic response to lighting environment changes and camera movement Detailed photometric variation for modeling rain streak illumination in [GN06] Offline rendering with a particle system using a large pre-computed database of raindrop textures

9 Cinematographic Rendering of Raindrop Effects Techniques designed to fit a specific goal [BSS*04] for Matrix Revolutions Rendering rain particles via manuallyanimated and deformed instanced 3D rod-line shapes for water drops and splashes Commercial software (Maya and 3D StudioMax ) includes offline approaches for rain rendering based on particle-systems [ Her01] for Pixar s A Bug s Life used CFD to model water splashes Rendered via particle systems using implicit surfaces to model individual drops These methods generate detailed physics-based movement and accurate visual representation At the price of prohibitive computational speed Matrix Revolutions

10 Water Droplet Simulation and Rendering: Related Work Offline approaches dominate the field [KIY99] used particle systems in a discrete environment to simulate droplet movement on surfaces [FHP99] modeled flowing water droplets with a mass-spring system simulating surface tension and volume conservation [WMT05] presented physically-based method for droplet simulation using level set distance field to represent water droplet surfaces Simulate variety of physically accurate small-scale fluid phenomena [WMT05]

11 Our Contributions A comprehensive and flexible system with numerous effects Supports rendering rain effects in real-time in complex dynamic environments Provides a high degree of artistic control for the final look A collection of inexpensive and streamlined algorithms for photorealistic rendering of rainfall and rain-related effects Suitable for games and interactive applications A combination of image-space, particle-based and dynamic simulation algorithms

12 System Supports Dynamic lighting and animated objects Illumination due to lightning Atmospheric scattering, glow and fog Attenuation effects Halos around light sources and objects Flexible post-processing pipeline

13 Contributions: Rain Effects Algorithms on the GPU Image-Space Rainfall Algorithm Misty objects in the rain Raindrop splashes and halos around light sources and objects due to light scattering Simulation and rendering of water droplet movement on glass surfaces on the GPU Streaming water Atmospheric Light Attenuation View- GPU-Based Water Surface Simulation and rendering Dependent of Simulation for Puddle dripping raindrops Warped Ripples due to Splashes Reflections Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006

14 Rendering Complex Environment: Without Rain

15 Rendering Complex Environment: With Rain

16 The ToyShop

17 Rendering Rain Precipitation

18 Creating Rainfall with Multiple Layers of Rain Render composite rainfall layer prior to the final scene post-processing Rendered as a full-screen quad over the scene The artists specify the rain direction and speed in worldspace to simulate different rainfall strength Initial raindrop distribution is simulated with an animated 8 bit rainfall position placement texture To simulate raindrop mistiness we blur the rain layer using the post-processing pipeline

19 Moving Per-Frame Raindrop Distribution For each pixel we determine each raindrop s distribution position in clip-space Using current time step t and the rainfall speed v r Sample from the raindrop distribution texture Artists specify a rain parallax parameter p r Maps the depth range for rain layers Model multiple layers of rain in a single pass with a single texture fetch from the rainfall position placement texture Using p r, r i, and the screen-space raindrop location (x i, y i ) p r * r i used as the w for a projective texture read Allows simulation of raindrops falling with different speed at different layers

20 Rain Appearance The layer of rain is shaded by using a normal map of varied individual raindrop shapes Light the raindrops using multiple scene lights Reflective and refractive model incorporating Fresnel and transparency The rain must correctly respond to lightning illumination Use lightning brightness parameters to bias reflection and refraction for water effects Lightning brightness also adjusts the opacity and the glow amount Our approach does not require any preprocessing and can handle an arbitrary number of light sources Simulate motion blur by dynamically modifying LOD based on raindrop traversal speed During a projective texture fetch into rain distribution and normals textures

21 Raindrop Particles Rain To simulate raindrops falling off various objects in our scene, we used billboard particle systems Artist-placed and animated based on a template system For each individual particle: Motion parallax: control raindrop velocity as a function of its depth Motion blur: stretch billboard based on velocity Use a normal map for a droplet (a blurry version of a full raindrop s normal map) Tangent space is defined by the view matrix Only compute reflection and refraction to simulate air-to-water transmission Droplets should appear more reflective and refractive when the lightning strikes Biased lightning brightness value adjusts the refraction / reflection color Raindrops off objects

22 Raindrop Splashes Raindrops splash when they hit solid objects We simulated that effect with individual particles colliding with various objects Directly collide with in-game objects or special collider proxies Used a filmed high-quality splash sequence for a milk drop: Raindrop splashes We used just one splash sequence for thousands of particles Randomly scale particle size and transparency Randomly flip u texture coordinate based on pre-specified particle random color Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006

23 GPU-Based Water Simulation for Puddle Rendering

24 Water Ripples in Puddles Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006 Goal: Dynamic realistic wave motion of interacting ripples over the water surface With fast simulation directly on the GPU Water ripples are generated as a result of rain drops falling onto the geometry in the scene We support generation of raindrop ripples as a result of direct collision However, for our scene that would require too much memory Instead, we use a stochastic seeding method for the simulation Seeding rain drops into a texture Spatter raindrops as points into the water simulation texture Can also render object outlines into the seeding texture to generate wakes

25 Water Surface Response Treat water surface as a thin elastic membrane Ignore gravity and other forces Only account for surface tension At every time step, infinitesimal sections of this surface are displaced Due to tension exerted from their direct neighbors Acting as spring forces to minimize space between them

26 Computing Ripple Heights Vertical height of each water surface point can be computed with partial differential equation: 2 z t = v 2 z 2 2 x + 2 z 2 y 2 where v is the velocity of the waves traveling across the surface Or using simplified Navier-Stokes equation (see [SMT04]) Solve this equation in real-time to determine water wave height for each point in the lattice PDE solution is computed with explicit Euler integration in SM 2.0 pixel shaders Two passes gave us sufficiently stable simulation During the final pass, compute the water normals using a Sobel filter on the wave heights

27 Integrating Water Puddles We render a single water simulation for the entire environment All objects with water puddles sample from that (for example, streets, rooftop ledge, etc) Sample from water membrane simulation using current position Use the (xz) world-space coordinates for look-up Scaled by the artist parameter to vary by size of membrane simulation (and size of ripples) To reduce visual repetition, we rotate these coordinates by a pre-specified angle (ex: 15 ) Angle is specified per-object No additional geometry is required for water puddles

28 Water Droplet Animation and Rendering on Glass Surfaces in Real-Time on the GPU

29 Water Droplets Trickling Down on Glass Surfaces Adopted an offline raindrop simulation system from [Kaneda99] to the GPU Dynamically animate and render a large number of water droplets on glass surfaces on the GPU The simulation is modified to use a gather pass in the pixel shader, rather than original scatter-based particle system implementation The droplet shape and motion is influenced by the forces of gravity and the interfacial tension forces, as well as air resistance

30 Droplet Movement Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006 The surface of the glass is represented by a lattice of cells Each cell contains: The mass of water in each cell Velocity in x and y The droplet traversal quantity within the cell Water affinity parameter for the surface The droplet information is packed into a 16-bit per channel RGBα texture Surface tension is controlled by micro properties of the glass surface and presence of water at any given point

31 Droplet Movement Simulation Droplets can flow into one of the 3 cells below New cell to flow into is chosen probabilistically Biased by velocity, friction based affinity and wetness of the target cell Droplets have a greater affinity for wet regions of the surface Velocity is updated based on target cell chosen

32 Droplet Simulation: Forces F down F up = M * G = f static or f dynamic Gravity and mass are used to compute the downward force on the droplet Static friction for stationary droplets, and dynamic friction for moving droplets is used to compute the competing upward force The static and dynamic friction varies over the surface of the glass V new = V i +F/M * t This resultant force is applied to the initial velocity to determine the new velocity value for the droplet

33 Droplet Rendering After simulation, each cell contains a new mass value A bump map is derived based on this mass Used to perturb reflection & refraction vectors The droplet mass is also used to render dynamic droplet shadows of the simulation onto the objects in the background If the droplet mass is large enough, we render a quasicaustic highlight in the middle of the shadow for that droplet Eurographics Workshop on Natural Phenomena, Vienna, Austria, 2006

34 Results Discussion

35 Test System: ATI ToyShop Implemented in DirectX 9.0c Using HLSL SM 3.0 shaders Lua scripting language for rendering scripts Roughly 300 shaders for various rain-related components ~500 individual shaders for the entire system Although our environment is a night time scene, the algorithms can be applied in variety of lighting simulations Environment geometry, textures and related offscreen buffers used ~240MB video memory High resolution textures were used to capture extreme detail of the represented world 256K 600K polygons and 0 22K particles per frame Approximately 5K-20K particles were used for raindrops and rain splashes

36 Performance Measured on 1GB Dual 3.2 GHz Pentium 4 PC with ATI Radeon X1900 XT with 512MB of video memory Achieve frame rates of fps Rendering time for the raindrop particles and splashes was limited by the CPU

37 Rain Components Rendering Times Composite rainfall Misty object halos Raindrop splatters Effect Raindrop particles (5-10K) Raindrop splashes (5K) Post-processing for glow GPU-based water rendering (with simulation) Rendering objects with droplets (includes simulation) View-dependent reflections Complete system rain rendering Rendering w/out rain effects 243 fps fps 52 fps fps 285 fps fps fps fps fps 32 fps fps Frame Rate Eurographics Workshop on Natural Phenomena, Vienna, Austria, ms ms ms ms 3.50 ms 2.41 ms 6.98 ms 6.59 fps 8.74 ms fps ms Rendering Time

38 Summary Convincing and visually pleasing rendering of rain effects enhances the realism of outdoor scenes in many applications We described a comprehensive system for interactive rendering of rain effects in real-time in complex environments Presented a number of novel effects, including Rainfall rendering GPU-based water simulation for dynamic puddle rendering Water droplet animation and rendering using graphics hardware View-dependent wet reflections All of the these effects help us generate an extensive, detail-rich urban environment in stormy weather

39 Acknowledgments John Isidoro for developing many of the effects described here and his great work on the ToyShop demo Thorsten Scheuermann for the initial GPU water idea

40 The ToyShop Team Lead Artist Lead Programmer Dan Roeger Natalya Tatarchuk Artists David Gosselin Daniel Szecket, Eli Turner, and Abe Wiley Engine / Shader Programming John Isidoro, Dan Ginsburg, Thorsten Scheuermann and Chris Oat Producer Lisa Close Manager Callan McInally

41 Reference Material All of these presentation and related materials Downloadable publications and videos from ATI Research Tatarchuk, N Dynamic Parallax Occlusion Mapping with Approximate Soft Shadows, ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games Tatarchuk, N., Isidoro, J Artist-Directable Real-Time Rain Rendering in City Environments. Eurographics Workshop on Natural Phenomena, Vienna, Austria, September 2006.

42 Questions?

43 Thank you!