Horde3D http://horde3d.org/forums/ |
|
Shader system updates http://horde3d.org/forums/viewtopic.php?f=8&t=580 |
Page 1 of 1 |
Author: | marciano [ 11.12.2008, 23:24 ] |
Post subject: | Shader system updates |
Back in the days when I had to write some more advanced shaders for my final university project, I realized that the current system has some limitations which make it not very convenient to use for more complex shaders. So I started experimenting with some new ideas. We finally decided now that it would be a good idea to take them over to Horde 1.0, even though it is a major change in the way shaders are written. The updated shader system has three new main features:
Here is an excerpt from the new model.shader which is some sort of standard shader used for geometry. It looks much nicer with proper syntax highlighting (I have a style definition for Notepad++ if anyone is interested). Code: [[FX]] <!-- // ================================================================================================= Model Shader Supported Flags: _F01_Skinning _F02_NormalMapping _F03_ParallaxMapping // ================================================================================================= --> <Context id="ATTRIBPASS"> <Shaders vertex="VS_GENERAL" fragment="FS_ATTRIBPASS" /> </Context> <Context id="SHADOWMAP"> <Shaders vertex="VS_SHADOWMAP" fragment="FS_SHADOWMAP" /> </Context> <Context id="LIGHTING"> <RenderConfig writeDepth="false" blendMode="ADD" /> <Shaders vertex="VS_GENERAL" fragment="FS_LIGHTING" /> </Context> <Context id="AMBIENT"> <Shaders vertex="VS_GENERAL" fragment="FS_AMBIENT" /> </Context> [[VS_GENERAL]] // ================================================================================================= #ifdef _F03_ParallaxMapping #define _F02_NormalMapping #endif #include "utilityLib/vertCommon.glsl" #ifdef _F01_Skinning #include "utilityLib/vertSkinning.glsl" #endif uniform vec3 viewer; attribute vec2 texCoords0; attribute vec3 normal; #ifdef _F02_NormalMapping attribute vec3 tangent, bitangent; #endif varying vec4 pos, vsPos; varying vec2 texCoords; #ifdef _F02_NormalMapping varying mat3 tsbMat; #else varying vec3 tsbNormal; #endif #ifdef _F03_ParallaxMapping varying vec3 eyeTS; #endif void main( void ) { #ifdef _F01_Skinning mat4 skinningMat = calcSkinningMat(); mat3 skinningMatVec = getSkinningMatVec( skinningMat ); #endif // Calculate normal #ifdef _F01_Skinning vec3 _normal = calcWorldVec( skinVec( normal, skinningMatVec ) ); #else vec3 _normal = calcWorldVec( normal ); #endif // Calculate tangent and bitangent #ifdef _F02_NormalMapping #ifdef _F01_Skinning vec3 _tangent = calcWorldVec( skinVec( tangent, skinningMatVec ) ); vec3 _bitangent = calcWorldVec( skinVec( bitangent, skinningMatVec ) ); #else vec3 _tangent = calcWorldVec( tangent ); vec3 _bitangent = calcWorldVec( bitangent ); #endif tsbMat = calcTanToWorldMat( _tangent, _bitangent, _normal ); #else tsbNormal = _normal; #endif // Calculate world space position #ifdef _F01_Skinning pos = calcWorldPos( skinPos( gl_Vertex, skinningMat ) ); #else pos = calcWorldPos( gl_Vertex ); #endif vsPos = calcViewPos( pos ); // Calculate tangent space eye vector #ifdef _F03_ParallaxMapping eyeTS = calcTanVec( viewer - pos.xyz, _tangent, _bitangent, _normal ); #endif // Calculate texture coordinates and clip space position texCoords = texCoords0; gl_Position = gl_ModelViewProjectionMatrix * pos; } [[FS_ATTRIBPASS]] // ================================================================================================= #ifdef _F03_ParallaxMapping #define _F02_NormalMapping #endif #include "utilityLib/fragDeferredWrite.glsl" /> uniform sampler2D tex0; #ifdef _F02_NormalMapping uniform sampler2D tex1; #endif varying vec4 pos; varying vec2 texCoords; #ifdef _F02_NormalMapping varying mat3 tsbMat; #else varying vec3 tsbNormal; #endif #ifdef _F03_ParallaxMapping varying vec3 eyeTS; #endif void main( void ) { vec3 newCoords = vec3( texCoords, 0 ); #ifdef _F03_ParallaxMapping const float plxScale = 0.03; const float plxBias = -0.015; // Iterative parallax mapping vec3 eye = normalize( eyeTS ); for( int i = 0; i < 4; ++i ) { vec4 nmap = texture2D( tex1, newCoords.st ); float height = nmap.a * plxScale + plxBias; newCoords += (height - newCoords.p) * nmap.z * eye; } #endif vec3 albedo = texture2D( tex0, newCoords.st ).rgb; #ifdef _F02_NormalMapping vec3 normalMap = texture2D( tex1, newCoords.st ).rgb * 2.0 - 1.0; vec3 normal = tsbMat * normalMap; #else vec3 normal = tsbNormal; #endif vec3 newPos = pos.xyz; #ifdef _F03_ParallaxMapping newPos += vec3( 0.0, newCoords.p, 0.0 ); #endif setMatID( 1.0 ); setPos( newPos ); setNormal( normalize( normal ) ); setAlbedo( albedo ); setSpecMask( 0.1 ); } At the moment the new format is completely orthogonal to the old one, so old shaders compile without any problems. The changes can be found in the svn repository. Please tell us your opinion on the updated system... |
Author: | AcidFaucet [ 12.12.2008, 02:55 ] |
Post subject: | Re: Shader system updates |
Looks interesting. In your example, would "FS_SHADOWMAP" and "VS_SHADOWMAP" typically also be present in the shader file? |
Author: | Vurlix [ 12.12.2008, 05:38 ] |
Post subject: | Re: Shader system updates |
small bug in CodeResource::assembleCode() Code: - std::string &depCode = _includes[i].first->assembleCode(); + std::string depCode = _includes[i].first->assembleCode(); I find it interesting that MSVC would accept the original as valid.. Other than that, it looks good. |
Author: | Volker [ 12.12.2008, 12:19 ] |
Post subject: | Re: Shader system updates |
Vurlix wrote: Code: - std::string &depCode = _includes[i].first->assembleCode(); + std::string depCode = _includes[i].first->assembleCode(); I find it interesting that MSVC would accept the original as valid.. It is indead a little bit confusing, since you get a copy from the assembleCode method returned but store that copy as a reference. Not sure what will be the result when compiling this with MSVC but your solution seems to be the more correct one. I fixed it in the SVN. Thanks for the hint. |
Author: | swiftcoder [ 12.12.2008, 16:51 ] |
Post subject: | Re: Shader system updates |
The new system looks very nice, and fixes pretty much all of my gripes about the previous system. Good work! |
Author: | attila [ 12.12.2008, 22:55 ] |
Post subject: | Re: Shader system updates |
It looks great, (I haven't updated the code because I modified horde3d's image loader so it will be a little work to merge the changes. And hoping for dds support in SVN ) A shader cache would be also great but AFAIK you can't compile glsl to binary. Has anyone checked d3d11 ctp? It seems like nextgen hardwares will support interfaces to handle the multiple combinations of shaders. Also d3d11 supports cards with shader 2.0 support and above. Maybe I will give a try and create a d3d11 port... Keep up the good work! |
Author: | marciano [ 13.12.2008, 01:51 ] |
Post subject: | Re: Shader system updates |
Great to see that you seem to like the update @AcidFaucet: The other contexts are also in the file, the example is just an excerpt. @Vurlix: Right, according to that link it is not ISO 99 standard. But a const reference should work and be more optimal since it avoids an additional heap allocation for the string. @attila: Unfortunately a shader cache is not possible for the reason you tell: GLSL has no precompiled binary shaders |
Author: | Vurlix [ 13.12.2008, 03:22 ] |
Post subject: | Re: Shader system updates |
As far as I know, you can't return references or pointers (const or not) to stack objects. Or is there an exception to this rule? One workaround would be using a string reference passed as parameter for the output.. |
Author: | swiftcoder [ 13.12.2008, 04:01 ] |
Post subject: | Re: Shader system updates |
marciano wrote: @Vurlix: Right, according to that link it is not ISO 99 standard. But a const reference should work and be more optimal since it avoids an additional heap allocation for the string. Named Return Value Optimisation ought to take care of this without any effort on your part, so a straight copy should be just as efficient as the const reference.
|
Author: | Manny [ 06.01.2009, 12:05 ] |
Post subject: | Re: Shader system updates |
This looks good, I just have one question. If you have 3 materials, A, B and C, with flags defined as: A: _F01_Skinning, _F02_NormalMapping B: _F02_NormalMapping C: _F03_EnvMapping After all shader combinations are compiled, how many shadow shaders will you end up with in this instance? Presumably it will compile shadow shaders for each combination, but these will actually be the same as the shadow shaders do not use any of the flags. Does it do anything internally to remove these duplicate shaders or will the application end up using 3 different shadow shaders for these materials? |
Author: | marciano [ 07.01.2009, 23:17 ] |
Post subject: | Re: Shader system updates |
Manny wrote: After all shader combinations are compiled, how many shadow shaders will you end up with in this instance? Presumably it will compile shadow shaders for each combination No it should not. Each shader context knows the flags it is using. This flag bit mask is anded with the requested combination mask. So if no flags are used in a context, the context mask is 0 and the result is combination 0 regradless of what flags are requested. More generally speaking, maybe it should be explained that the flags have a special naming scheme: _F<digit><digit>_<ReadableName> (e.g. _F01_Skinning) The ReadableName is just for convenience and competely ignored by the shader system, solely the number identifies the flag. So _F01_Skinning is the same as _F01_Whatever. |
Author: | marciano [ 15.02.2009, 16:11 ] |
Post subject: | Re: Shader system updates |
We finally integrated the last part of the new shader system. The changes are: - Uniforms need to be declared explicitly in the shader FX section now. This makes it easier for programmers and tools to track which uniforms are used. But the real advantage is that uniforms can have default values now. - The current texture unit configuration got replaced by sampler declarations. You are no longer required to keep track of texture unit indices and can just assign a texture to a arbitrarily named GLSL sampler uniform (also declared in the FX section). This is much more convenient especially when using material links or complex pipelines where the texture unit setup is scattered over several files. - Some texture related parameters (coordinate address mode and filtering) are defined in the FX section now and no more in the materials. This is similar to the way it is done in d3d fx and cgFX and makes the whole system cleaner. Unfortunately the changes make it impossible to continue supporting the previous shader format. So you will need to port/convert your own shaders when Beta3 is out. This is actually not really hard but nevertheless, here some notes to make that task easier: Code: Shader Porting Notes
- Create a new file with the extension .shader - Add a new section [[FX]] at the beginning - For each context $CCC in your old shader - Create new code sections after the FX section for the vertex and fragment shaders, e.g. [[VS_$CCC]] [[FS_$CCC]] Please note that with the new system it is possible to share code sections between contexts, so if the vertex shader is the same for all contexts, it is enough to have a single code section for it. It is also not required that the shaders have VS or FS as prefix. This is just a suggestion. - Copy the old shader code into the new sections - The content of the <DefCode> blocks can just be copy&pasted - The <InsCode code="$CODE"> blocks simply become #include "$CODE" - Create a new XML Context element in the FX section: <Context id="CCC"> <Shaders vertex="VS_CCC" fragment="FS_CCC" /> </Context> - Copy the XML RenderConfig node of the old Context to the new Context - Declare all of your custom uniforms in the FX section. You can give them default values if desired. - Add sampler declarations for all used custom textures. You are no longer required to name the GLSL samplers tex0 to tex11 but can have an arbitrary name. Please take a look at the sample shaders, they show quite clearly how things are done now. - Be aware of the fact that you can use #ifdefs now to write an übershader. This can help to merge many different shader files into a single one. |
Author: | marciano [ 21.07.2009, 22:23 ] |
Post subject: | Re: Shader system updates |
Another update to the shader system: The XML based FX syntax got replaced by a syntax that is similar to CgFX. I think this is a good upgrade. The FX sections look a lot more readable and familiar now. Many tokens are slightly different from CgFX but the basic syntax structure is the same (including annotations). Since CgFX / D3D FX is the de facto standard for shaders, this change should make it easier and more convenient for people to get started with Horde shaders. Example: Code: // Samplers
sampler albedoMap = sampler_state { Address = Clamp; Filter = Bilinear; }; // Uniforms float4 myUniform < string UIDesc = "This is my uniform"; > = {0.5, 0.5, 0, 0}; // Contexts context AMBIENT { VertexShader = compile GLSL VS_GENERAL; PixelShader = compile GLSL FS_AMBIENT; } |
Author: | marciano [ 07.12.2009, 23:37 ] |
Post subject: | Re: Shader system updates |
It is possible now to specify a default texture for a sampler. This makes the material system more robust since it is no longer required that all samplers are configured in a material. If for example no normal map is specified, a default flat one is just used without any visual artifacts. It is also easy now to make the model shader support a single color value since the color uniform would just get multiplied with the default white texture if no albedo map is specified. |
Page 1 of 1 | All times are UTC + 1 hour |
Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |