Horde3D

One of the things in Horde that should to be improved rather soon is the handling of resource attribute updates/queries and dynamic resource creation at runtime. The challenge is to create a nice and generic interface for this so that we don't need to clutter the API with dozens of very specific functions.

One option is to divide resource data in elements, parameters and streams. Elements have an index (to get the n-th element of a type) and represent, for example, samplers and uniforms in the material resource. Parameters are the attributes of elements. Finally, there is streams which contain raw binary data.

The API could look like this:



And some enums:



This would work for most of the horde resources. My only concern ist that it does not work well with resources that have hierarchical data, e.g. the pipeline. The pipeline has stages which contain commands. It would be possible to address the stages as elements but the commands could not be accessed. Ok, that would not be a real problem for pipelines since I would consider them as immutable (fully static) anyways but I could imagine that particle systems could cause similar issues once they get a bit more advanced in the future (e. g. particle groups).

Maybe, does anyone else have an idea for a different concept for the resource API?

I'm not saying that these are a much better ideas, but maybe worth considering.

Context based API like the OpenGL the matrix manipulation: first setting the matrixmode than manipulating the selected matrix. This way you can even traverse a hierarchy.

The other method could be to expose sub resource handles. But to make the ids unique could be tricky (maybe an int64 with a 32bit resource and a 32bit subresource index).

A better particle system could be useful. For now, we are considering to use http://pyro.fenomen-games.com/ in our project. It has a great editor (without an editor a complex system isn't really useful). I'm planning to create an extension for it.

Another suggestion that might be worth considering:

What about moving the entire resources system out of the Horde3D core? That way, one could easily implement one's own resource types. You could, for example, easily add support for CgFX or use a model file format without having to convert it into Horde's binary format first.

Also, it would be possible to replace Horde's resource management mechanism with a custom one that better fits the application's needs (like loading resources on a background thread).

I just wanted to say thanks Nicolas for looking into the resource/data issue There have been a lot of times with Horde where I have decided not to go ahead with some feature because passing procedural data to the engine was too much effort. These API extensions would really be helpful!

I'm not a big fan of Context based APIs like attila mentions, but in this case it may actually be the best way to allow generic hierarchies without having a bloated API.
Another option would be for "sub-resources" (e.g. a pipeline stage) to have their own resource ID. E.g. a stage's resource ID would be an element of a pipeline, and data belonging to the stage would be an element of that ID.You'd still need a mechanism for passing all of that data into horde itself (so that horde can upload the data into the GL), which is what marciano is trying to define here I think.
Perhaps for an idea like this to work, you would basically split Horde into two layers. The lower layer is just a thin OpenGL wrapper, and the higher layer implements resources/scene-nodes etc... This could be a nice idea, but probably doesn't fit with Horde's short term goals.AFAIK this is already possible - the file loading code is implemented in the utils library, which is really just an example of how to make a horde app.

Yes, but it's not possible to upload the resources to OpenGL on another thread. Wait... does OpenGL even support that?

I guess it is about time that this gets solved
If you have a special use case let me know so that we can see if the API is powerful enough.


I thought exactly the same when I saw it and in the end I like attila's idea because it is completely orthogonal to the proposed API. We just need to add some function pushResourceEleme( res, elem, index ) to push a parent element to the stack. For non-hierarchical resource data the stack can stay empty, so there is no API overhead involved.


One of the longer-term goals is to make the loadResource function thread-safe. OpenGL does not allow asynchronous resource uploading, so we need to queue upload queries for efficient synchronization.

I'd be really interested in this as at the moment I've been writing Verse enabled applications and assembling a framework around the verse api as an asset pipeline. At the moment I'm using the Quel Solaar renderer to explore scenes but it is still quite unstable and a pain to integrate C++ code without a rewrite. I'm starting to look through the Horde3D code base and familiarizing myself with it so i can see how this would be possible.

A basic version of the new resource data interface is available now in the svn repository.

The following functions were added:

The enums indicate the type now and look like this:

As always, feedback, criticism and improvement suggestions are welcome!

Could you please provide an example on how to iterate over all vertices of a mesh node (in index order) with this new system?

Iterating over the vertices/indices is not difficult: you get the number of indices/vertices with PI_GeoIndexCount/PI_GeoVertexCount. To access the actual index/position data, you just map (mapResourceStream) the index/position streams (S_GeoIndexData/S_GeoVertPosData) and cast the pointer to the appropriate format. The format is float for the position data and uint32 or uint16, depending on the value of PI_GeoIndices16, for the index data.

Here is an idea to further simplify the resource API: the element could be removed and derived from the parameter in which it is implicitly contained anyway.

This would simplify the code for exchanging the first texture in a material from

to

There is a reason that the elements were introduced and that is to make it possible to add or remove elements. However, I don't think that we will support that anyway. I prefer the idea that the resource structure can only be defined at creation time and only the values can be changed later (i.e. everything which does not involve a memory allocation). For a texture for example, that means that you can change the pixels but not the width, height or texture format. This helps to save memory on critical platforms (no std::vectors which get too large because they are reallocated with twice the size), helps to avoid memory fragmentation (no additional reallocations) and is better for performance since no hidden/implicit allocations are happening.