Horde3D

Using dynamic lights and shadows, Horde3D is pretty good at real-time diffuse and specular. But lately I've been thinking about real-time ambience (which brings us close to GI) and I haven't had time to test my ideas, so I thought I'd share them to see if they make sense :wink:

These ideas are formed in the following context:

• A mostly indoor environment, like a lot of FPS games, consisting of rooms/corridors/etc...
• All direct lighting is rendered using Horde's current lighting systems
• Ambient lighting is approximated using 'light probes' that are used to render cube-maps of the rooms. It is up to the level designer to place one or more probes in each room at good locations.
• I'm thinking mostly about low-frequency bounced light here, not highly detailed stuff. So SSAO could be used as well as this for even more detail.
• I'm assuming a deferred renderer (because it makes things easier), but it could probably work with a forward renderer too.
• This works best for levels where there isn't multiple floors, thought it could be extended to support that.

Here's an example map layout with a bunch of probes laid out:

(image hosting by flickr)

Part 1 - getting and storing the ambience

Each probe renders the scene around it to a low-resolution cube-map using a full pipeline supporting shadows etc. Not all cube-maps have to be updated every frame - you could gather the n closest probes to the camera, and update one of them each frame.

After a cube-map is updated, a fragment program is used to convert the low-res textures into spherical harmonics (e.g. the first 9 SH coefficients). The outputted SH values represent the incoming light at that probe's location (although much more compressed then the cube-map - only 9 float3s instead of, say, 64*64*6 float3s!).
These values are saved into an SH lookup-table (RGB texture), where the texture's columns represent the coefficients, and the rows represent the probes.
e.g. probe 3 would write it's SH outputs to UV(3,x) in the texture, where x is the coefficient number.
So if we had 32 probes and were using 9 SH coefficients, this texture would have to be 9x32 pixels.

Row #0 would probably be kept black for use later - this way probe #0 can be a special reserved probe that you can use if you don't want any ambient light.

After setting all of this up, we've got the whole scene's ambient (bounced) lighting stored in a nice little texture.

Part 2 - using the ambience
When we render the final scene (not the cube-maps), we want to add this ambience during the lighting passes. Thanks to the G-Buffer, we know the position of each pixel (and importantly, the normal), which we should be able to use to select which row(s) of the SH lookup-texture to sample.

For this we need another lookup-table! The probe-location lookup-table (RGBA texture).

This lookup-texture is basically made from a top-down view of the level (like my picture above). Each texel in the image represents a small section of the world, and the values in the texture represent which probes should be sampled by pixels in that section.

Seeing as textures are addressed in normalized coordinates from 0 to 1, I'll call the positions in this texture (i.e. the probe-location lookup texture) "normalized world coordinates".

A tool would probably be required to generate this lookup-table, although it could be done by hand...
Basically, some of the channels in this texture represent probe ID numbers, while other channels represent blending weights.
E.g. The R channel represents the closest probe, and G represents the 2nd closest probe. B and A are the blending weights for those two probes.
Or...
R, G and B are the ID's of the closest 3 probes, and then packed into A (3 bits: 3 bits: 2 bits) are the blend weights.

Here's an example of the data that could go in the R channel (the ID of the closest probe):

And the G channel (the 2nd closest probe):

As you can see, I've ignored probes #5/6/7/8 because it's very tedious to make these textures ;)
Also, I've used high-contrast shades of grey so that the data is visible. In a real texture those shades of grey would be almost the same.

So finally, the ambient lighting shader would:
1) sample the position from the G-Buffer
2) Sample the normal from the G-Buffer.
3) convert to "normalized world position" by offsetting/dividing by the world's bounding box.
4) Use the XZ of the "normalized world position" as a texture coordinate to sample a texel from the "probe-location lookup-table" [important - no texture filtering here!]
5) Use the texel's R value as the V coordinate, and the numbers [0-8] (+1/2 a texel) as the U coordinate to sample 9 texels from the "SH lookup-table".
6) Use the SH values and the normal to calculate the main incoming light.
7) Repeat steps 5 and 6 using the G value to calculate the 2ndary incoming light.
9) Use the texel's B and A values to blend the main and 2ndary incoming light values together
10) Add this value to the frame-buffer


I've never done anything this complex before, so let me know if you think it's feasible, or if you've got any suggestions.

Isn't this roughly the technique used in Half-life 2, and several games since then?

_________________
Tristam MacDonald - [swiftcoding]

The cube-maps, yes. Although usually the cube-maps are static, not dynamic. Also, in HL2 AFAIK they're used as reflection maps for gloss (still as cube-maps, not SH), not for pure ambience.

In all the probe based systems I've seen, the choice of cube-map/SH-data is per-object or per-vertex. What I want to do is choose the appropriate probe per-pixel.

E.g. a moving character could potentially be using a different probe-texture on each pixel of their model...

Cool, this sounds like a really interesting idea for lighting dynamic objects. It could also work for outdoor scenes where you have dense forests or places with much occlusion. Usually it would probably be possible to tag some light probes as static (when you know lighting won't change) to improve the performance. The only practical issue I see at the moment is that Horde has currently no dynamic render-to-cubemap rendering but that should be easy to implement.

I was wrong about this - HL2 does use 1px cube-maps for ambience, as well as the high-res cube maps for gloss. The ambience isn't dynamic, and is applied per-object though.
Any render-to-texture support would be ok. I'm told that Dual Paraboloid Maps might be more appropriate than cubemaps anyway and they would only require regular render-to-texture.