Horde3D

This is actually a sequel to the thread about ambient lighting improvements.

The latest idea for improving the material system is to introduce a new material link concept. A material could link to another material which can define additional or overwrite existing texture stages and uniforms. This would make it possible to define e.g. an ambient cube map at a single place instead for each material. Basically implementing this should not be difficult, it is just a matter of making the setMaterial function recursive.

But one thing about which I am concerned is performance, especially for the uniforms. Currently uniforms can have an arbitrary name. The problem here is that the uniform always needs to be found in the currently active shader which is probably a relatively expensive operation. Of course we could use some caching of the uniform locations but there would still be some overhead coming from hash tables which map the uniform name to the uniform index in the shader. Furthermore the new link system would make the caching much more complicated.

A simple solution to these problems would be to have predefined uniforms, similar to the texture units. We have sampler uniforms tex0 to tex11 which can be configured in the materials and accessed in the shaders. The same could be introduced for generic vec4 uniforms, so that we have e.g. uniforms unif0 to unif15. The access to the uniforms would consequently work by an index which is of course much faster than using generic names.

Another advantage of such a restricted uniform system is that the engine can do some checks which could be very useful for larger projects. The engine can see if a shader uses a uniform and output a warning if that uniform is not defined in the material. Such situations usually lead to unexpected behavior which can be difficult to debug.

Well I'm not quite sure. Should we have the nicer system with arbitrary uniform names or the more performant and robust one with uniform indices? Please tell me what you think!

I would be strongly against using pre-defined names. In fact, even before reading this I was going to suggest loosening the restriction for texture names. Using "tex3" or "unif12" makes a shader a lot less readable, and throws away some of the advantage of using a high level shading language instead of assembly.

I think putting more effort into the lookup system is the way to go. There are a couple of different schemes we could try for optimally mapping human readable names to resource indices, and I'm sure we can find a good one. Furthermore, I'd suggest extending the scheme to the texture names!

Edit: If performance for dynamic updates turns out to be a problem, we could give access to shader parameters via handles, so that a search only has to be done once to get a handle. I don't think this necessarily needs to be done at first though, and could co-exist with access by name.

Basically I see this the same way as you, that's why I am so unsure here. The worst drawback when using named uniforms is that Horde can't check if the materials define all textures and uniforms used in the shader. Well not so easy at least, of course we could still parse the shader ourselves to find uniforms.

Having named texture uniforms would be nice but personally I don't consider it to be as important as named vector uniforms. If we introduce them, do you suggest that the user still assigns the texture unit or do you think that Horde should do that automatically (which could be some more overhead)?

I have implemented the basic link system now. At the moment there is no caching used. I still have some points on my TODO list that university needs in the next time, so I have to give them priority (software skinning, finishing resource cloning, stereo rendering, etc.). If you are interested don't hesitate to play with the material system code to make it better. I probably won't have time for that in the next few weeks...

We can enumerate the shader attribs and uniforms with glGetProgramiv(GL_ACTIVE_{ATTRIBUTES|UNIFORMS}) and glGetActive{Attrib|Uniform}, and use this to check that everything has been set, and uses the right type. Textures/samplers are also enumerated by glGetActiveUniform.

I think that Horde should probably assign texture units automatically. It's not something that the user should have to worry about if we can take care of it for them.

Ah ok, I didn't know these functions. This solves the problem of course...

The caching system for uniforms is really crucial. I just removed the glGetUniformLocation from the recursive terrain rendering function and got a speedup from 150 to 230 fps in release mode...

I just implemented a useful new feature: pipeline stages can also have material links now, so you even don't need to link the global material containing the ambient parameters in each material but only one time in the pipeline.