«Natalya Tatarchuk 3D Application Research Group ATI Research What’s in It for You? • Share our lessons of developing an extensive environment for ...»
Real-Time Rain Rendering
in City Environments
3D Application Research Group
What’s in It for You?
• Share our lessons of developing an extensive
environment for next generation
– To help you in similar research and development goals
– The new technology developed for this demo
• Learn what it means to render rain in city environments
– Novel algorithms
– Lighting and lightning
– Raindrops and puddles
• Lots of eye candy!
• Rain rendering: introduction
• Related work
• Lightning system
• Rendering rain precipitation
• Water effects – puddles and droplets
• Post-processing for glow and mist
• Wet reflections
• Results discussion Introduction The Challenge of Rendering Rain
• Rain is hard!
– Creating convincing impression of rain in rich natural environments is a difficult task
• It is a complex phenomenon – Vast diversity of rain components – Huge amount of small details
• But - rain rendering greatly enhances outdoor scenes!
– Many applications in games and motion pictures – Challenging to film and even more so to render at interactive rates Photorealistic rain greatly enhances the scenes of outdoor reality, with applications including computer games and motion pictures. Creating a faithful representation of rain in complex natural environments is a non-trivial problem. The challenge stems not only from the visual complexity and diversity of the rain components and scene objects, but also from the huge amount of small details that should be modeled to obtain realistic visual effects and physically plausible simulation. However, rain rendering greatly enhances outdoor scenes and is an important problem for computer graphics, with many applications in computer games and motion pictures.
Filming rain scenes involves a significant effort and cost due to complicated setup.
This task becomes even more challenging when trying to create photorealistic rendering of rain in rich environments at interactive rates.
Creating an Illusion of Rain in Games
• Rain consists of numerous effects interacting together
• Previously games focused on rendering rain with one or two components – Often only using simple alpha-blended particle systems for rain which follow the camera – doesn’t look very realistic – That type of rain does not respond to dynamic lighting – Requires too many particles for a feeling of strong rainfall
• Recently, some were able to incorporate our rain effects directly into their games • “Heavy rain” by Quantic Dream – post-processing rain effects and water droplets Rain is an extremely complex atmospheric natural phenomenon. It consists of numerous visual effects interacting together. Some recent games which incorporate rain rendering use simplistic approaches, including rendering stretched, blended particles to simulate falling raindrops or using blended animated textures (as in [WW04]) to render precipitation. These methods fail to create a truly convincing and interesting rain impression. Furthermore, games often limit using only one or two individual rain effects (the rain particles or the scrolling textures and perhaps a CPUbased water puddle rendering) to simulate the impression of rainy environment. This results in an unrealistic rendering with the rain not reacting accurately to scene illumination, such as lightning or spotlights. Simply rendering particle rain is not enough: Rain has a variety of visual cues, not just the streaky drops. Missing pieces can destroy the illusion of immersion Contributions
Rain is an extremely complex atmospheric natural phenomenon. It consists of numerous visual effects interacting together. Although research has been done in the area of rendering some individual components (rain streaks in [GN06], [WW04], or water droplets on surfaces in [KIY99], [WMT05]), there exists a gap for creating a complete system for rain rendering in complex environments. Simply adding several individual components is insufficient as the discerning viewer’s eye quickly notices the missing elements. For a truly convincing illusion of rain environment we must present a coherent system supporting the full gamut of the natural phenomena associated with rain.
There is a strong need for inexpensive and streamlined algorithms capable of photorealistic rendering of rainfall and rain-related effects in games and interactive applications. Photorealistic rain rendering requires convincing display of rainfall and raindrops, various dynamic water-related effects for puddles and streaming water, and a variety of scene effects for atmospheric effects and wet materials. Our main contribution is an intuitive, comprehensive and flexible system for photorealistic rendering of rain effects in real-time in a complex environment. We provide a high degree of artistic control for achieving the desired final look. To our knowledge, this is the first complete system of this kind.
Novel Algorithms for Rendering Individual Rain Effects Post-Processing Rainfall Algorithm
We present a number of novel algorithms for rendering the individual components of
rain, including the following:
• A new post-processing composite rainfall algorithm exhibiting raindrop shape perturbations and dynamic response to varied illumination conditions and viewpoints
• Simulation and rendering of raindrops dripping from various objects in the scene
• Techniques for raindrop splashes and splatters on solid objects and in water puddles
• An engine-driven lightning illumination system for simulating lightning flashes
• Halos around light sources and objects due to light scattering in rainy mist
• A novel effect for rendering view-dependent warped reflections on wet surface materials and puddles using reflection impostors
• Atmospheric light attenuation
• GPU-based water surface simulation for puddle ripples due to raindrop splashes
• A novel approach for the simulation and rendering of water droplets on glass surfaces on the GPU, with wetting, droplet merging and separation phenomena
• A large number of supporting effects resulting in increased scene realism Rendering Complex Environment: Without Rain Rendering Complex Environment: With Rain Demo: The ToyShop Let’s take a look at the demo.
Related Work Related Work in Rain Precipitation
Rain effects have been examined in the context of atmospheric sciences, as well as in the field of computer vision. However, at the moment only a few approaches exist for creating realistic rainfall rendering that dynamically responds to the lighting environment and camera movement. Constant brightness rain strokes are generated in [SW03] for simulation of rain in videos. This approach fails to represent dynamic illumination and camera movement. In [WW04] rain and snow precipitation was modeled with several interpolated hand-drawn textures with constant brightness mapped on a double-cone which is dynamically aligned to match the camera orientation in real-time.
Related Work Reference
• For a thorough review of related work in rendering rain components, look in – Tatarchuk, N., Isidoro, J. 2006. Artist-Directable Real-Time Rain Rendering in City Environments, in proceedings of Eurographics Workshop on Natural Phenomena, Vienna, Austria 2006 Lightning System There’s No Thunder W’out Lightning
• Lightning and Thunder increases the feel of a rainy, stormy night
• Illumination from the lightning flashes needs to affect every object in the scene
• Uniformly aligned shadows are crucial
• At the same time, we are still using shadow mapping
• Computing lightning shadows for each additional lightning light can hurt performance A dark night in rough weather would not affect the viewer in the same manner without the sudden surprise of a lightning flash followed by the inevitable thunder. Creating a realistic lightning effect in interactive applications is challenging for several reasons.
Illumination from the lightning flashes needs to simultaneously affect every object in the scene. Uniformly aligned shadows are crucial. Simply adding extra shadowing lights for each lightning is still an impractical approach for interactive applications due to associated performance cost and additional memory requirements for storing shadow maps.
• Lightning is a strong directional light that has to affect every object in the scene
• Lightning effect would feel very repetitive if it only comes from one direction
• The viewer can get very close to some of the lightning shadows… – Resolution has to hold up Creating a convincing and realistic lightning effect is challenging for a variety of
•Lightning is a strong directional light that has to affect every object in the scene.
•Lightning effect would feel very repetitive if it only comes from one direction.
•The viewer can get very close to some of the lightning shadows… resolution has to hold up.
•Needs to work seamlessly with the other lighting solutions in our scene, such as our depth mapped shadow casting lights.
•Computing this type of lighting effect at run time would be a huge performance hit.
Lighting System Solution
During the preprocessing phase for the environment several key lightning source directions are picked by the artists and global illumination solutions are computed for each selected light source. The encoded illumination value (in a series of 8 bit textures per direction) is used at rendering time by all rendering components to modulate the illumination result to account for a lightning flash event. This compact representation allows us to create consistent uniform lightning shadows.
Implemented lightning as 2 unique lightning lightmaps:
- One at an angle and one closer to directly above the scene (slightly offset)
- Artist editable intensity parameter to allow for custom mixing of these two maps.
Mixing these two maps in different ways makes it seem like we have a wider variety of lightning direction than we actually do. Pre-rendered in Maya and stored as a 8bit grayscale image. Only needs to be 8bit as it is only used as an intensity value multiplier of the underlying HDR lightmap. Does not need any additional tone info.
Packed into R and B of a single RGB image to reduce draw calls.
This value is also piped into other shaders, the rain for example. Used lightning intensity to adjust the opacity of the rain. Use either or both lightning brightness parameter or lightning lightmap sample added to the regular lightmap sample before tone mapping Incorporating Illumination from Lightning
• Every shaded pixel uses lightning illumination information
• The script propagates these to all object shaders – A lightning brightness parameter – Lightning illumination is added to the regular illumination for each material prior tone mapping – Negligible performance cost: cheap and effective • 1 additional texture fetch plus a couple of ALUs
• All object materials use this model and appear to respond accurately to lightning illumination
• Translucency of water is affected by lightning strength – Used the lightning brightness value to adjust the pixel’s opacity as well for rain effects Rendering Rain Precipitation Rain Precipitation
• Consists of water drops falling at high velocity, refracting and reflecting the environment
• Falling raindrops create the perception of motion blur and parallax – Dynamically generate ripples and splashes in puddles
• A hybrid system: an image-space approach for the rainfall and a particle-based effects for dripping raindrops and splashes – Render individual raindrop shape variation and motion parallax due to differing depth for raindrop movement – Render dynamic raindrop illumination
• Image-based precipitation effect does not incur extra performance overhead for modeling heavy versus light rain – Unlike purely particle-based approaches Precipitation due to rain consists of spatially distributed water drops falling at high velocity, refracting and reflecting the environment around it. As the raindrops fall through the scene, they create the perception of motion blur and parallax and generate ripples and splashes in puddles. We developed a hybrid system of an image-space approach for the rainfall and particle-based effects for dripping raindrops and splashes. We render individual raindrop shape variation and motion parallax due to different depth for raindrop movement as well as dynamic raindrop illumination. Unlike purely particle-based approaches, the image-based rainfall precipitation effect does not incur extra performance overhead for modeling heavy versus light precipitation.
Rendering Multiple Layers of Rain with a PostProcessing Composite Effect Our image-space rainfall effect simulates multiple layers of falling raindrops in a single compositing pass over the rendered scene.
This method differs from most previous approaches in rendering the rainfall without the use of a particle system with a large number of particles or rain textures. We provide a set of artist controls for the rain direction, velocity, and strength. The raindrop rendering receives dynamically-updated parameters such as the lightning brightness and direction from the lightning system to allow correct illumination resulting from lightning strikes.
Creating Rainfall with Multiple Layers of Rain
• Render composite rainfall layer prior to the final scene post-processing – Rendered as a full-screen quad over the scene
• Must consider performance implications – Every pixel on the screen goes through the rain shader
• Need minimize distracting repeating patters for rainfall – All the while keeping shader complexity minimal We render a composite rainfall layer prior to the final post-processing of the rendered scene. We must consider the practical performance implications of the rainfall layer as a full-screen pass and design the algorithm to yield pleasing visual results without expensive computations.