Horde3D

Sounds reasonable, but see next point.

SOIL has a separate function SOIL_direct_load_DDS_from_memory() which handles loading the compressed image directly into a texture. The default loading behaviour just decompresses it to normal image data.

My intention for the moment was to go with the default behaviour, and then let Horde re-compress it on upload. But in retrospect, this doesn't make so much sense. I can detect the ST3C compression in the image loader, and pass the data on unmodified to the renderer to upload, but I need to be able to request compression from the renderer.

Should we add an additional (possibly only internal) flag to the renderer, something like ResourceFlags::DDSCompression, or should I give the renderer the ability to detect ST3C (also simple)?

I would agree with this change.

_________________
Tristam MacDonald - [swiftcoding]

I would propose the following way:

- Create a new file utImageDDS.cpp / .h
- Add a DDS loading function based on the code in SOIL_direct_load_DDS_from_memory(); best would be an interface similar to stbi just for code beauty
- This function checks the dds header and returns false immediately if the "file" is no valid dds; the function is called in the image loader before the stbi functions
- If we have a valid dds we store the data in the output of the image loader and upload the compressed data directly to the renderer in the texture resource loading functions; for this we will need to extend the texture upload functions with some additional format parameter and also the image loader routine

I think DDS files should never be uncompressed and the allowCompression flag should be completely ignored for them (there wouldn't be any advantage because the dds already have the compression artefacts).

Open questions:
- Do we want to support dds cubemaps or use the vertical cross as usual
- Should it be possible to resize dds to a POT size (would be slow and probably require compression/uncompression code, so probably rather not)

Sounds good, I will have a bash at implementing.
What about hardware that doesn't support S3TC? (if there is any GL 2.0 class hardware that doesn't support it)
DDS cubemaps are output by many tools. Supporting the vertical cross means decompressing.
How much 2.0 class hardware is there without at least basic support for NPOT textures? This perhaps better as part of the art pipeline, to avoid artefacts.

_________________
Tristam MacDonald - [swiftcoding]

Ok, so we have a variety of DDS files, and it looks like a lot of code paths to support them. This is going to be fun
We can have any combination of the following:

• Compressed/Uncompressed
• 2D/Cubemap
• With/Without mipmaps.

It looks like uncompressed should be handled exactly as we handle texture currently, but compressed have to go through to be uploaded by the renderer. Should we ignore existing/auto generate missing mipmaps based on the user specified mipmap flag?

_________________
Tristam MacDonald - [swiftcoding]

Texture compression is part of the GL core since 1.3, so that's no issue for us


I think it won't be so hard/much work to implement the mentioned dds features. I have seen some code on the net for dds loading that is quite slim. If the dds has mipmaps, I would just use these and disable automatic generation.

Great to see you working on that feature. And please don't feel in a hurry for implementing it, quality is clearly more important here than having it tomorrow

Don't have numbers to compare speed but this http://developer.nvidia.com/object/texture_tools.html may be of interest to use. It's MIT license so i think it is compatible with Horde3D. you probably already know about it, but here it is anyway

This is my first post, I just want to congratulate for your great work. I hope the community will grow larger. I think there should be more samples/tutorials, but the existing docs are great.
I hope it will be continued despite the seemingly slow progress. Is there a roadmap for further development?

We are using Ogre3d for our hobby project but I'm considering using this engine instead of ogre. I like its simplicity.
But for us it lacks some features:
- dds support. So i just wanted to know if there is any progress with this. I don't know anything about OpenGL's texture compression speed, but I think it is lot faster to load pregenerated compressed textures with mipmaps. (I know Id software does realtime texture compression for megatexture but that is another story...)

- Direct3D support (I'm addicted to NvPerfHUD ). I read the topic about this, but I think graphics engine should support both API. As far as i can see only renderer/rendererbase files depend on OpenGL(and of course the shaders). So this maybe wouldn't be too much work.

- And some small things that could be easily added (eg.: stencil states)

Sorry for the long (and little offtopic) post

The project is definitely continued (check the svn) but progress is unfortunately a lot slower since Volker and me both have daytime jobs now. But that's not so bad since we have most of the features we always wanted already integrated. I think in the past Horde was evolving exceptionally quick and now it is rather on a standard open source speed See OGRE and Irrlicht, they also don't release so many major versions per year.

We can't give a concrete roadmap at the moment but what we plan for the future are improvements to make the engine rock stable and better suitable for larger projects.

I'd actually like too see some more "open" development. Like using bugzilla/trac or something like that for bugs/issues/improvements etc, so that everybody knows about it who might be able to fix it.
EDIT: Of course something like this might need at least a general roadmap partially. -.-

Are you working with Volker? Or in a diff company? I know that Volker (via PM) is not doing any game related work in his new job, just curious if you guys found or applied to jobs in the 3D or game programming industries that are very strong in Germany.


I agree with Pheonix64. I think it would improve manageability of the project. We now have 4 Developers in SF.net, but we don't really know who is doing what, or for that matter all others that are doing their own work (like the managed resource manager from Pheonix64, etc...) something like bugzilla or Trac might help to get a clearer picture.

- DDS and PowerVR texture support are the most relevant to me.
- D3D backend. I think there is someone working on this. See viewtopic.php?f=1&t=346

I think it might not be a bad idea to have a list of features that people are in need of or would like to have. Combine that with volunteers to work on those features and see what we can come up with roadmap wise.

By PowerVR texture support do you mean support for GL_IMG_texture_compression_pvrtc? I have that working outside of horde, because I needed it on the iPhone.
While trakc or the sf bug tracker might be good in the long term, why not start a wiki page in the meantime? It will give us a good place to check off functionality as it is implemented.

I haven't had time to finish adding compressed texture support to the engine, because it means major changes to the way textures are currently loaded. In light of the crazy way the Terrain sample loads height maps, I think image and texture handling needs a bit of discussion before I progress.

_________________
Tristam MacDonald - [swiftcoding]

1) Yes, and adding ETC would also be nice. Are you doing a iPhone/ OGL ES port?

2) I agree. That is probably the easiest thing to do to get things started. So here is the Community Roadmap wiki page. Needs some table styling work and content.

See the relevant roadmap page: Compressed Texture Support.

_________________
Tristam MacDonald - [swiftcoding]

Now i am kind of confused do we discuss in forum or in wikipage?
Anyway...

Regarding your open questions.



Usually there is also hardware support for S3TC (or a subset of the 5 formats) it is just more efficient to use PVRTC or ETC in those platforms. If the developer is going to use textures at least one of these formats "has to be" (usually is) present in hardware, a software fallback may not be necessary, i would defer this effort for another time if someone needs it.

The ability to query what formats are available at runtime is a very nice feature, i would like to have it, even if most of the time the developer will know ahead of time what formats the platform can handle and will accordingly compress the image data using the appropriate methods.



This is a tricky question, without knowledge of a specific context. In presence of multiple options the support for the fastest and support for high-quality settings provides a nice duality. The thing is, in certain contexts the one format that balances both features is usually optimal (or not, but if it's not available you can't really tell, tricky).

Support for all formats and containers, would be ideal. However, prioritization by fastest/highquality on a first phase would be enough for most use cases!?

---
I also have a few discussion points.

Regarding mipmap generation - should it always be done ahead of time, in the art pipeline? Or should there be support for generated mipmaps at all levels (this means additional extensions, and additional work).

Is there a need to support a heavier image library like freeimage? Is this desirable?

LOL - lets keep it to the forum.

Most OpenGL ES1.1 hardware doesn't have ST3C, although that may change as ES 2.0+ becomes common.

This requires that the developer use a separate art pipeline for each target device - fair enough.

As a specific example, Apple and PowerVR both provide a wrapper format for PVRTC, and a toolchain to produce them. Although they use the same compression, the wrapper format is non-compatible, and supports slightly different features. The Apple tool runs only on Macs, while the PowerVR tool runs only on Linux and Windows.

Speed/quality isn't really an issue. However, I do wonder how people would feel about rolling this out of the engine entirely?

I am talking about building a Horde3D-specific (though the format would be open, of course) wrapper format, which can hold all types of compressed/uncompressed image data (uncompressed, RLE, JPEG, S3TC, PVRTC, ETC, etc.). There would be a tool that leveraged FreeImage (or similar) to convert/batch convert other image formats into the Horde format, which would move this whole issue out of the engine and into the content pipeline.

I have a feeling it is too big of a break from tradition to be well received - but it is the approach taken by the WildMagic engine.

All levels, I am afraid. if you are I/O bound, you want to generate them on the fly, CPU bound you want the pre-calculated. You may need to tweak them manually in the content pipeline, or you may need to modify the quality at load-time.

FreeImage doesn't offer anything apart from .PSD support, over what we have already - and it doesn't have mipmap generation or DDS support, last I checked.

_________________
Tristam MacDonald - [swiftcoding]