Horde3D

I have no particular need for it, since all my targets feature S3TC, so a software fallback is indifferent for my particular needs.



I have never used a Mac platform, nor developed for the iphone. I also was not aware that that they provided a PVRTC toolchain/wrapper, I hope it is possible to abstract the differences and use a unified set of functions to work with PVRTC textures.


I don't think i understand what you mean. Can you provide more details on how this woul work? what are the benefits and issues?


DDS is supported. I am not advocating for freeimage, just putting a question in the open regarding the need for a more advanced image manipulation libraries, DevIL or any other that does the job right is ok by me. personally i have no need for them.

We have had a couple of requests for both ETC and PVRTC, so someone must need them.

it is possible to abstract, but it multiplies the code we have to maintain.

We strip all texture loading code out of the engine, and replace it with a custom format, specifically designed to contain any type of compressed/uncompressed image data.

Then we provide a tool that will use FreeImage (or whatever) to convert standard image formats into our custom format.

Pros: simplifies texture handling in engine, allows support of *any* format with minimal work and no complexity on the engine side, simplifies art pipeline for different device targets
Cons: requires separate tool - can no longer drop an image in and it works, you now have to convert the image first.

_________________
Tristam MacDonald - [swiftcoding]

Personally I need support for the formats i don't need the software fallback.


Does Apple change the wrapper allot? I know that PVR does not make many changes.



Hum... i don't get how it would simplify the art pipeline, i would still need to have a file for each platform, one compressed with pvrtc and one with s3tc... Maybe i am not getting the concept. It looks like allot of work, and is a fundamental change to how the engine handles images, i kind of like to have this kind of stuff at a low level, and not abstracted...

The idea is to make the pipeline the same on each platform - right now you need completely different tools on each platform.

Mainly, it simplifies the enigne code itself. Just the code to handle DDS is a lot of added cruft, and by the time we add ETC, and 2 variants of the PVR wrapper, it is getting ridiculous.

The current scheme is also fundamentaly at odds to Horde's philosophy - we don't try to support every 3D model file under the sun (.obj, m.3ds, .x, ...), instead we provide a converter and a single format that exactly matches our needs.

_________________
Tristam MacDonald - [swiftcoding]

I don't know the solution. But I think a simple cross-platform asset pipeline looks something like this:
(I know that many engines has plugins(material/exporter/...) for DCC apps but that is a lot of work)

Every data (model, texture) is stored in 'raw' format.
3d Model/animation are in collada /fbx/... textures in psd/png/hdr/...

For every resource/platform there is a description how the data should be converted. (eg. model vertex compression/ animation compression, texture - format, mipmap count/generation params). It can be implemented like a 'make' or something similar build program and can be made user friendly with some GUI.
There could be even more output version for the same textures (eg on PC for cards with/without 3dc texture compression).

So I don't think that the same asset would be good for every platform(not supported compressed format is a pain to handle). Especially if the loading times are important and for 'not so powerful' plaforms like ipod.

I like the idea of having a horde3d texture format, but without a viewer (plugin?) it would be a pain to use.
For a new format I think dds would be a good starting point. I think with new FOURCC's it could be extended to support the mentioned formats, but maybe this would't be the best option. It would be great if there were some other well established graphics card friendly image format...

Offtopic : I have implemented a different resource loading mechanism (async loading is work in progress). And I'm again impressed by the design of the interface. In Ogre I have to create multiple listeners to modify the resource referencing and it seemed so hacky. Here I've added a preprocess step for the xmls and I think it is much cleaner (maybe a little slower but the cost will be only some ms for a whole map).

Then I was thinking about that I could easily add dds support. Simply convert the dds to ?tga? (or something) in the loader. But of course I couldn't upload precompressed textures and mipmaps. But if the engine had a binary texture format(this could be dds) that would support compressed formats/mipmaps/cubemap then a loader would convert data to that format. This way the current image loading lib could be moved from the engine to the utils. And it would be much easier to add new formats. This would also make the horde3d simpler.
In our project probably I would only include the dds support and drop the stbi library.

I've read the forum and saw that Marciano wrote about something similar.
What do you think?

http://icculus.org/physfs/ I am using this for file loading

Even if you support a custom format, it's still in best interests to still support JPEG, and TGA (or some other alpha supporting format) for the reasons of hastened development. Yes plugins can be written for graphics software, but converter dependent lines are crude, and automating can be a butt.

I was just talking about an internal custom texture format. But If someone has a pipeline that has a converter for textures (like for meshes from dae) he/she could just use this format for storing data.

The Horde3d could support other formats(jpg,tga,dds) but image support could be in Horde3DUtils.dll instead of Horde3D.dll. If someone uses the Horde3DUtils's resource loading they wouldn't have to change anything.

In this way a new image format(pgf,jpeg200,...) could be added without modifying the core engine, which is a pain in an open source engine.

but it's just my 2 cents

That would create a dependency on Horde3DUtils. A utility library should not be a requirement.

I agree with attila. IMHO core engine doesn't need to know about different formats (also limiting i/o operations to the memory would be good)
But well that would make Core dependent on image upload functions. Smaller core & thicker shell VS bigger core & thinner shell. Probably just a choice of design

Not all applications need those functions, so not a dependency in general. For instance, I use procedural synthesis for all my textures, so all I need is an upload function.

_________________
Tristam MacDonald - [swiftcoding]

The core engine has to submit that data therefore it must know a good deal about it, unless you'd propose to write a single proprietary image format, which would be a bad idea, topping or even competing with most existing formats is an unrealistic goal (they each have their pros and cons).

At the same time, a solution for wider format support should also consider the fact that solely using XML for other resources is silly, at some point optimal binary archives for release builds would be ideal, especially in circumstances where streaming is a necessity, the same approach used for handling image after image should at its core be suitable for dealing with similar issues elsewhere.

We only need a very restricted set of formats - compressed/uncompressed, 1-4 channels, with/without mipmaps. That would be very easy to package into a custom container format, perhaps easier than abusing DDS to support other forms of compression.

Agreed. To my mind, the core engine should only know how to load binary data from memory, and the current XML resource manager should be factored out into a separate (optional) layer.

_________________
Tristam MacDonald - [swiftcoding]

I totally agree with you. It should support cube texture( and maybe 3d texture) too. DDS uses format (Eg. D3DFMT_A8R8B8B8, D3DFMT_R16F, D3DFMT_L8, D3DFMT_DXT1) for describing the compressed/uncompressed and 1-4 channels part, something similar could be used in horde3d's format with additional support for PowerVR and other formats.



This could be great also, but for me it is not high priority. The data heavy resources(geometry, animation, texture) are in binary format. I know commercial engines that uses 'text' for similar data, because of flexibility and ease of development/debugging. But for loading large maps this could be a bottleneck...