Horde3D

Hi Swift. You can use the emulator for development (the emulator that i use for a few years is PowerVR SDK it's a opengl desktop wrapper for opengl es 2.x functionality). You can also use the 3rd gen ipod, it uses the same processor and its cheaper than an iphone 3gs.


Cool! I keep looking at Qt as a solution for all cross-platform development needs it's a really great platform, but it's a huge dependency to add.



That is ok. I really appreciate all the work you guys have put into the project, it keeps getting more and more polished. I just mentioned it because i remembered it from a long time ago, and it's a topic that i am interested on ever since i read AMDs Sushi Engine approach. I moved to Unity for iphone programming, it's not perfect but it works good enough. I stopped working on my own opengl es 2.x engine and on porting horde3d there, now i am just focused in content and doing gameplay programming. But i always keep an eye on Horde3D

I'm curious now about the idea of porting horde to OpenGLES 2.0. I was just reading bits here and there about desktop opengl to opengles conversion and noticed the issues were more to do with code using immediate mode rather than using buffer objects, but horde itself only uses buffer objects and glsl shaders to do the grunt work yes? Just asking as a newbie here but would one just remake the egRenderer, egRendererBase & utOpenGL parts of the code to match the ES2.0 API? Of course you'd remove the "is X function available?" parts or rather split it to support PVR SGX quirks, etc. Actually another question, it looks like you guys reimplemented what GLEW or GLee are made to do with utOpenGL, was there a reason for this or is it just to keep dependencies down?

What sparked my interest is that I plan to order an N900 nokia phone which runs linux/x11 for the OS (maemo 5) and noticed it uses a PowerVR SGX under the hood

N900 + horde + python = potential epic win

-Alex

_________________
-Alex
Website

Yep you pretty much have to rewrite the renderer classes with openGL ES API 2.x calls. Then there is some plumbing issues in utils but no biggy. Take care of any variable size issues and minor platform issues, just iteratively debugging, a couple of days to polish it up. The biggest problem is porting the shader/material system, that will take a bit of work! And there are some differences in shader languages, and several limitations, you also have to optimize textures with PowerVR format and tools if you want to have good speed... so it's kind of tough to solve all this with just a different build branch.

For start you may want to take a look at PowerVR SDK and examples and apples iPhone SDK.

Before i discovered Horde3D i build a very simple engine based on the PowerVR SDK sample framework. It worked but it wasn't very advanced at the time there was no Unity for iPhone, or Shiva or Torque, so going custom was the only option. At the time i tinkered with the idea of porting Ogre3D or Irrlicht, but compared to H3D these engines are bloated, i wanted something simple but powerful and H3D is it! After i discovered Horde3D,I decided to start porting it, but i wanted to have a hardware platform to debug any performance issues, emulator are good and all but in the end the hardware is where it counts. So I decided to wait for OpenPandora but that project has suffered constant delays. My goal at the time was to simply push a menu in the editor and pick build for Desktop, build for iPhone and be done with it. But then Unity for iphone came out and it's pretty decent, and since i am not in it for the tech even though i find it interesting, i decided to focus on gameplay and actually getting a game out the door.

(regarding OpenPandora, i am still waiting for mine to be delivered, so i bought a beagle board but at the time there where no drivers for the GFX so you couldn't use the hardware acceleration, but with the ipod touchs and iphones 3gs now being available it's all good! I dislike apple products because they are just expensive re-branded PC parts and it sucks that Apple does not let it's OS run in non-Apple hardware, but they do have a nice design and attract a crowd that is not afraid to spend money.)

Anyway, if anyone wants to take the lead and start the project i may be able to help from time to time (mostly weekends), but i am not going forward with it. I would really like to see and have a choice to pick Horde3D for use in iPhone. Currently there are OpenGL ES 1.1 for irrlicht and ogr3d, i read something about there being a irrlicht port for openGL ES 2.x but the shader system was not working, ditto for ogre3d (AFAIK). Writing the rendering code is not that hard, you just need to change APIs but ES 2.x is pretty compact and easy to port, the problem with H3D is that there is no fixed function so the shader/material system has to be working to be able to run the examples.

I recently did a project where we had to use a fixed-function GPU (no shaders! ), so as "the next best thing" we made a text file format that let you configure the texture-environment (multi-texturing unit), tex-coord generators, lighting channels, etc in data, instead of hard-coding different material types. We even called these text files "shaders" even though they weren't HLSL/CG/GLSL

Something like this might be handy for pre-GLSL phones.

Probably too much off-topic but getting back at the conversion to OpenGL ES 2.0, I was seeing that the debug and overlay drawing are using some immediate mode functions but then I noticed GL_MODELVIEW & GL_PROJECTION and friends are scattered around to make the lights, camera & shadow projections, and from reading the powervr migration doc from the SDK, ES 2.0 doesn't like it one bit and would rather that stuff be done in a shader

I think I'll wait for marciano to make those parts shader-driven for when horde becomes GL 3.x compliant

DarkAngel - I think that sounds just like how quake 3 did its shaders, pretty cool technique though for back then.

_________________
-Alex
Website

Moving the tranformations to the shader shouldn't be a big problem, should it?
Just add two additional matrix uniforms to the shaders and set them to the projection and modelview matrix. Then in the shaders it should be only necessary to multiply the vertex positions with those matrices. I currently don't have the time to test that, so no responsibility is taken for the correctness of this information.

yep that is pretty much it. here is a simple tut on getting started with opengl es 2.x shaders quick read but covers most of the steps. http://www.mobileorchard.com/getting-st ... phone-3gs/

Also there are a few limitations regarding shaders, like can't have multiple shaders in the same shader object, etc, etc... little adjustments here and there. Shaders have to be compiled and linked, and there are a few other limitations that don't exist on desktop ogl. Also some "important" features to achieve advanced effects, or performance improvments are available thru extensions, that may not be present in all phones. Though with powerVR being the biggest player thus far it is still not very relevant.

Do you know if the OpenGL part of the Snapdragon CPUs from Qualcomm is also from PowerVR? I thought it was an ATI implementation?

Originally Snapdragon was combined with an in-house part, later replaced by a part purchased from ATI (but manufactured by Qualcomm itself). Performance data is a little tricky to find, as are Qualcomm devices with a GPU

_________________
Tristam MacDonald - [swiftcoding]

Swift is correct. I however have no experience with Qualcomms Snapdragons development, only with ATIs emulator. I know that ATI sold it's handheld unit to Qualcomm, a guy i "know at AMD" changed jobs at the time. The ATI and PowerVR implementations do not (did not) work well with each other at least in terms of emulators. So i have always used the powerVR SDK, it's more complete, and i use their data formats for textures and effects in my engine, since it's basically their engine framework with tweaks for my needs.

Qualcomm devices are comming out now so it's definetly something to account for, in the future.

If anyone is ready to undertake the task the one book that i can recommend is http://www.opengles-book.com/ the authors are very approachable. And it's available as Ebook and Kindle format. Anyone can read it in a weekend and get a solid foundation on OGL ES 2.x. Most of the code in the book is a guide in writing a Utility Framework to deal with porting OGLES 1.x (or any other fixed-function pipeline) to OGLES 2.x:



Here is a sample chapter covering the platform differences from Win Emulators and iPhone SDK (and iphone peculiarities) along with
http://www.opengles-book.com/OpenGL_ES_ ... hapter.pdf

Doing the port is not very hard. It's just very time consuming, due to the amount of tweaks, adjustment and platform considerations that you have to take into account.

Good stuff, was it for low-spec devices?

Mobile platforms like their "food pre-chewed", that is a very flexible way to use the fixed function pipeline, but if it has to be processed at run-time then it may not be the best option for a fixed function fallback.

I am terribly sorry to keep making off topic posts.

But i have been catching up on Qt 4.6 and they really made some great additions to this version. It's a powerhouse of a framework, Nokia was very smart in acquiring them.

Anyway one of the things that caught my eye is that they now have an API Qt/3D to do certain 3D tasks. This actually solves in a cross platform way a few issues that have been mentioned in this thread. Plus It's very OpenGL /ES 2.x friendly. It provides it's own basic math library with useful functions like:



Plus they setup a nice framework to work with both GLSL and GLSL ES 2.x shaders. This is actually something i was working on at the time i dropped development of my hacked engine, and that i thought contributing to Horde3D. It's nice to see a public API that saves the porting work for Horde3D.


Yes it can. Thanks Qt for taking care of it for all of us.

Their approach however comes with a BIG caveat that may be a problem for Horde3D to adopt.

Ya see the reason why i never fully committed to doing the port at the time was due to the need to also port all the shaders. The reason why no open source 3d engine actually has a usable opengl es 2.x renderer working is exactly this. The need to port all the shading code to GLSL/ES and to maintain it.

So adopting this approach would require that further Horde3D shader development moved to this approach (if it was to run the same on both desktop and mobile). This is something that may be a deal breaker for wide adoption, since it's easier to find GLSL code and examples than GLSL ES code. Meaning that desktop developers could still use GLSL but if you want your code to run on both Desktop and Mobile then you have to write your code in GLSL ES. It's a trade-off, so Marciano and Volker would have to make that decision moving forward.

I have to do a little bit more research and try to find the time, but i am getting excited with the new possibilities that Qt 4.6 brings to the table. If anyone with more time could take the lead in porting it would be GREAT! I promise to help as much as i can.

Since shaders are only resources that can be loaded at runtime, I don't think it would be a big problem if we have separate shaders for desktop and mobile systems. Horde3D should be capable to run both depending on what rendering backend has been selected. So if Horde3D is running the ES renderer, you load the GLSL ES shaders and on desktop systems you load the desktop renderer with it's shaders.

In a perfect world, you can detect the system automatically and have shader implementations that results in the same rendering on both systems. Of course that means that you are limited to the capabilities of the ES platform you have. But I don't think that we should go the Qt way to try to merge both shader languages to one, to avoid rewrite of shader code. For common code parts you already have the Code resources we can include in shaders (or isn't that possible in ES shaders? ).
For this case the deprecated resource directories became useful again, because you could just change the resource directory for shaders to something like DESKTOP or MOBILE and don't have to duplicate materials only for loading another kind of shader.

Another problem is that we have to add some kind of compiled shader resource for the ES backend, but that shouldn't be a problem, should it?

I have to admit that I haven't read anything about shaders yet, so all I know is what marciano has provided with the sample implementations in Horde3D and what I've learned from discussion about them.

That would work. My issue is that as an Indie developer working in a real world project, i would definitely not like to have two or more sets of shaders and have to maintain them, so all my design work thinking about the port was with the intuit of having the least amount of work managing assets and shaders.


Well GLSL ES has different qualifiers, and the languages aren't really fully compatible. So the amount of "common code" is not as common as one would hope for. There are a few conversion tools http://sourceforge.net/projects/hlsl2glsl/ that help but that's adding extra tools to the pipeline. Something i try to avoid like the plague.

This is one source of struggle with the design of the port. Do we simply extend the engine and use different shaders, like for instance openscenegraph is doing. Use different code branches and create a tool to compile Code resources for each and have the runtime pick the correct option.

Then my brain goes into abstract more and tries to come up with anwsers to stuff like: How to accommodate to different backends if we port to DX9/10/11, how to support different languages and geometry, compute shaders... then there's the OpenGL 3.x context that also removes fixed function and requires that you use uniforms instead of the GL matrix commands, like ES 2.x, etc... So now you have manage (X) shaders for every (X) backend. Would it be simpler to use Cg and parse to diff contexts.... there is allot of design work and design decisions to be done and quite frankly it's allot of work, and many intrusive operations, so if anyone was going to pursue this it would possibly be best to develop another branch. If you try to be as abstract and general as possible to accommodate all sort of backends, the job gets tough. A really super extensible system is possible but it requires a huge amount of work so it may be counter producent. I tried to do this sort of system once and lost 6 months with no real solution, and at the time only 3 APIs DX9, OGL 2.x, OGL ES 2.x. It's a huge task.

So adding just iPhone support is a much more manageable task. If we just pragmatically do the changes necessary to support it: replace the gl matrix code, use different shaders for each version Desktop or Mobile is not a huge problem and there are tools that can help to port and manage the differences. OpenSceneGraph has a list of mini tasks that covers many of the things necessary to port to ES 2.x http://www.openscenegraph.org/projects/ ... /OpenGL-ES

OpenGL 3.x actually has allot of stuff in common with ES 2.x (i read somewhere that the spec goals, like remove fixed function and make the API more compact to be able to use ES 2.x and OGL 3.x in close relation) so it may not be a bad idea to try and figure out how to support it as well. I actually now have a OGL 3.1 card so i may try and write something in OGL 3.x and see what similarities there are with ES 2.x


No, that is actually the easy part of the problem. If you read the book i mentioned a few posts ago it shows exactly how to build such a framework.

I mentioned the Qt/3D because they try to solve many issues in a abstract and general way. So if you want to latter add support for Android and other devices Qt provides the abstraction layer and API to do the changes with the least amount of work on H3D part, since they are the ones that add support for more platforms, windowing, UI, and other plumbing issues. But they are also in a very embryonic state. Perhaps by the time of Qt 4.8 this will be a viable option and for now we should just do what everyone else is doing and do a simple port, use two sets of shaders and then try to come up with a smart way to manage platform differences.