Horde3D

Sorry for the lack of updates in this thread.

A few basic changes were done that pave the way to the removal of the scene graph. As there seems to be raising interest since recently in getting this change, we will reconsider it. But as usual we are very short on time, so I can't commit on when anything would happen.

Thanks for the answer.
I've seen the scene graph traversal/node update decoupling in the SVN repo. Probably this is one of the mentioned changes.

This is great news.

I'm already thinking this could be a great benefit for the future especially for things like threading and/or job dispatch techniques. Having the scene graph in the utility library or as an extension could still benefit small projects where they could adapt or extend the reference to their own higher level hierarchy/entity system which is decoupled from the rendering.

Having the joint matrices live in a model resource tightly packed, there could be potential wins here for SIMD performance or perhaps offloading to OpenCL (maybe a utOpenCL.cpp/h is in order soon) for large scalable crowd animation.

With regards to keeping some sort of spacial graph internal would be good to have for culling like you said marciano. I'm thinking if someone would want to use their own culling system would they modify horde internally or should horde have enough hooks for the high-level application to tap into some primitives (like extract AABB's and test them with some software zbuffer render on another thread, or some PVS system) and then send back the result to set visibility? (Edit: Actually never mind I just saw that this was commented on already.)

_________________
-Alex
Website

So the entire hierarchy is to be done away with? Besides the raw performance increase, I can't see the logic behind this move. Doesn't this also negate the ability to attach nodes to bones?

Or we weren't able to do that in the first place.

I think raw performance is very important in realtime graphics. Better performance means you can use more bones in your characters, more mesh to render or use more detailed physics/ai in your game. So overall make your product better.


AFAIK marciano said that you can still attach nodes to bones.

The more I read about submission based rendering the more I think It is the right thing to do. I don't say that horde3d should
be that low-level, but it's low-level part should work like that for better parallelism. Nowadays even mobile phones have 2-4 cpu cores, so supporting parallelisms very important.

here are two papers I found:
http://www.bitsquid.se/presentations/flexible-rendering-multiple-platforms.pdf
http://www.plm.eecs.uni-kassel.de/plm/fileadmin/pm/publications/chuerkamp/Diplomarbeiten/Johannes_Spohr_thesis.pdf

Frankly, the current performance of Horde3D is not very good on the CPU. It is very easy to get Horde to consume 100% of your thread's CPU-time, leaving no CPU time left over for the actual game.
A big reason for this is the design of the "high level" parts of the renderer - such as the scene graphs. It would definately be a good idea to separate this layer into another utility library, and not have it as part of the Horde core library, so that it is possible to obtain better performance, but also possible for people to continue using the inefficient scene graph if they prefer.

So...regarding model nodes, we should add model resources.

But what about the other node types (particles, cameras, lights)? What's the idea for them (add also particle/camera/light resources)?

Any hints? This week I have some time to work it on...

I think marciano has done already some steps towards removing the scene graph. But I don't have any additonal informations about the current state and I guess it won't make any further progress in the next few months.

As I've been touching some Horde3D related code today and I've been asking myself: does this change imply, that all resources get "flat" (=no need for XML anymore)?

Another thing thats bugging me for quite a while is that I have to link Horde3D+OpenGL into to our server application - we're using Horde3D meshes for Bullet. This might be a great chance to split Horde3DEngine into Horde3DRenderer and Horde3DResources (or similar). What do you think?

Regards,
Christoph

Well the only resource that is XML based are the group and model nodes. Geometry, Shader, Animations, Textures aren't XML resources. So "all resources" is a bit misleading. I never really thought about concepts for making the scene graph flat. I guess there are quite a few papers that discuss the advantages and disadvantages of having no scene graph within the rendering engine. As a user of Horde3D I don't think that I will get rid of a scene graph in general, so maybe there are better implementation possibilities outside of Horde3D but having a scene graph for structuring and manipulating the scene is IMHO essential. But I'm currently not involved in any game developement, so I might be wrong here.


Well, I don't know what you are doing within your "server" application, but maybe it's possible to abstract or split the horde3d resource loading enough to allow compiling of only the resource specific code as a seperate library (just by removing all other rendering specifc files, so no separate libraries in the official horde3d release). But I won't recommend splitting it only for your specific application, if that means code bloat and making the code more complicate to understand. After all Horde3D is still a rendering library and not a resource loader for bullet.

One definitely needs a scene graph, regardless if its "flat" with controllers or "hierarchical" with parent-child relations, thats out of question. I was curious if this opens the possibility to delegate all "XML attributes" to our "entity-component system" - this would provide way faster live-updates to the user (less hard disk access, less file watching, less resource-synchronization).


You're right. I didn't check before writing: resources have dependencies to the rendering code. Refactoring some code out of Horde3DEngine/egGeometry.cpp seems like a good solution for me.

Not sure what you mean with that, actually the XML files are only one way to create a model or resource within Horde3D, you also can do that entirely using API calls or within an extension the way you like it and without using any XML at all.

A long time ago, I proposed something similar:

viewtopic.php?f=1&t=214&p=665#p665

BTW, I think that the "resource" topic is something that we definitely have to discuss about to make any progress in this area. I mean, again: what about all the scene node types (which are not models)? Should they become resources too? And if so, should we remove the XML definition in favor of "flat" (binaries) ones?

Let's take a look what node types we have:

Group nodes: Are only there to allow hierarchical transformation changes, because of the scene graph we currently have in Horde3D => no scene graph => no group nodes anymore, only reason may be group specific uniforms.
Model nodes: Are there to handle multiple mesh data using a single interface. => no scene graph => no more hierarchical transformations for mesh nodes, but maybe still a list of meshes used by one model, with direct access to the mesh resources used within the model
Mesh nodes: The most essential node. If not within a scene graph, it has to be still present in a flat list to allow rendering
Joint nodes: Transformations for mesh data. So in case of no scene graph, we need the transformations already multiplied into a flat list, that will then be applied onto the mesh nodes/instances.
Light nodes: Still needed even if there are no hierarchical transformations anymore
Camera nodes: see light nodes
Emitter nodes: dito

So what's the benefit of having no scene graph in Horde3D at all? One reason may be that we don't need the hierarchical traversal of the rendered nodes anymore, what may improve performance and maybe it's easier to optimize rendering calls to reduce state changes. But maybe the biggest benefit: we can let e.g. Bullet handle all the transformations and don't have to sync Horde3D's scene graph with another one maybe existing in a physics engine.


But after all I haven't looked into the Horde3D source quite some time and haven't read any game engine specific papers for several years now. So don't expect my comments to be fully qualified.