Expanded TSE procedural shader set?
by Tim Betts · in Torque Game Engine Advanced · 03/09/2005 (9:03 am) · 17 replies
I've recently created my first interior in Cartography Shop with textures and exported to DIF. I then added the relevent addMaterialMapping and datablock Material { ... } code and created normal maps for some my textures. I then dropped it into the TSE example using the World Creator and it just looks great with the per-pixel specular lighting.
So now I'm wanting to start using custom shaders (and I found that great thread on GG which spells out how to do that), but I say custom because I can see that at this point the procedural shader set in TSE is fairly short. I mean no disrespect to TSE, since the included ones are certainly very powerful on their own.
However, can we expect to see the next milestone of TSE to include an expanded set of procedural shaders? if so, which ones? or is the policy that it will always be left up to each developer to write their own shaders? For instance, I'd like to plug in a parallax shader of some kind, but others like bloom, wood, etc., also interest me and I'd prefer to wait for them if their coming along in any form for TSE and focus on modelling in the time being. They are only custom for as long as their unique attributes not part of TSE's shader generator.
For now, I'll proceed with trying to include my own parallax shader.
Thanks -- Tim.
So now I'm wanting to start using custom shaders (and I found that great thread on GG which spells out how to do that), but I say custom because I can see that at this point the procedural shader set in TSE is fairly short. I mean no disrespect to TSE, since the included ones are certainly very powerful on their own.
However, can we expect to see the next milestone of TSE to include an expanded set of procedural shaders? if so, which ones? or is the policy that it will always be left up to each developer to write their own shaders? For instance, I'd like to plug in a parallax shader of some kind, but others like bloom, wood, etc., also interest me and I'd prefer to wait for them if their coming along in any form for TSE and focus on modelling in the time being. They are only custom for as long as their unique attributes not part of TSE's shader generator.
For now, I'll proceed with trying to include my own parallax shader.
Thanks -- Tim.
#2
03/09/2005 (8:01 pm)
Thanks for the reply Brian. Exciting stuff. BTW, the new water is looking cool.
#3
05/25/2005 (6:25 am)
Tim, I noticed this post and it seems you did this a while ago but would you be interested in posting how you did this and the modifications you made. I am interested in doing the first part you mentioned with the procedural shaders and need some guidance on what to add and edit to make this work. If you could show what you did to the .map or .dif file to include the shaders and if that changed anything else that would be great. (Where the files are located would be good too) :)
#4
Unfortunately, I never did get around to fully understanding all the changes to convert a given shader effect into procedural support, so perhaps others can explain the bullet points that have to be considered.
If it's simply modifying a stock procedual shader by extracting values from another texture, then I would search the source code for all references to bumpTex or others and model after those. I did do some of this exercise, but I felt it comes down to exercise in iteractions between other procedural constructs. This is obviously not an issue as long as you never use procedural constructs in yourmaterial datablock that are not compatible with each other.
The conversion of an effect to a procedural will also no doubt require some (probably extensive at times) shader writing knowledge to correctly code the effect as a procedural, just so that it can even live happily with the other procedural constructs that it has to at least work in conjunction with. Obvious, but nevertheless an effort.
Anyway, I'm still a TSE and shader rookie, so that's about all the advice I've got. If you have a procedural addition in mind, I would gladly try to work through it with you. I'm also sure that many other TSE members would also jump on board. There's is obviously great time saving value in converting custom shader effects into procedurals.
I would also say there is an obvious great opportunity to sell a shader-pack for some shader expert out there. Afterall, who here, wouldn't buy such a pack today if there was such an animal. I know I would.
Cheers -- Tim
06/03/2005 (4:43 am)
Hey Kip - Sorry for the delay getting back to you. Unfortunately, I never did get around to fully understanding all the changes to convert a given shader effect into procedural support, so perhaps others can explain the bullet points that have to be considered.
If it's simply modifying a stock procedual shader by extracting values from another texture, then I would search the source code for all references to bumpTex or others and model after those. I did do some of this exercise, but I felt it comes down to exercise in iteractions between other procedural constructs. This is obviously not an issue as long as you never use procedural constructs in yourmaterial datablock that are not compatible with each other.
The conversion of an effect to a procedural will also no doubt require some (probably extensive at times) shader writing knowledge to correctly code the effect as a procedural, just so that it can even live happily with the other procedural constructs that it has to at least work in conjunction with. Obvious, but nevertheless an effort.
Anyway, I'm still a TSE and shader rookie, so that's about all the advice I've got. If you have a procedural addition in mind, I would gladly try to work through it with you. I'm also sure that many other TSE members would also jump on board. There's is obviously great time saving value in converting custom shader effects into procedurals.
I would also say there is an obvious great opportunity to sell a shader-pack for some shader expert out there. Afterall, who here, wouldn't buy such a pack today if there was such an animal. I know I would.
Cheers -- Tim
#5
That way all shaders would be treated the same i.e. as custom shaders.
06/03/2005 (6:29 am)
Actually I've been meaning to ask why there is a procedural shader set? Shouldn't there simply have been a set of standard shaders and then we could simply add to this library as we needed to? That way all shaders would be treated the same i.e. as custom shaders.
#6
Having a management system for this is essential: GG is looking way forward with this one.
That said, for the most part I hand write and manage nearly all the shaders I use currently, but that is only a few. The procedural system in place is not super shader-dev friendly (much more artist->material friendly). Having your shader worked out beforehand as a custom material it would be relatively easy to port it to the procedural system to allow flexibility for your artists.
06/03/2005 (7:32 am)
The procedural shaders are generated to prevent a combinatorial explosion in the # of shaders one may have to write. Newer shader games can have hundreds or possibly thousands of different shaders (for different level of cards, LOD) many very similar to each other with a few differences. Also, the procedural shaders allow hardcoding of parameters that may be passed in thru a register in a more generalized shader: a potential performance issue. Having a management system for this is essential: GG is looking way forward with this one.
That said, for the most part I hand write and manage nearly all the shaders I use currently, but that is only a few. The procedural system in place is not super shader-dev friendly (much more artist->material friendly). Having your shader worked out beforehand as a custom material it would be relatively easy to port it to the procedural system to allow flexibility for your artists.
#7
06/03/2005 (11:08 am)
Look thru the shaderP000, shaderP001, shaderP002, etc hlsl files in the shaders folder. Those are the procedural shaders generated by your last execution of TSE. Your best bet would be looking to see how these are structured and looking thru the code in ShaderFeature.cpp/h.
#8
06/03/2005 (11:11 am)
@Tom - Is there a way to know that ShaderPXXX was generated by which material name?
#9
Why would you have different shaders for different levels of detail? I've never come across this before.
06/04/2005 (3:41 pm)
@JamesonWhy would you have different shaders for different levels of detail? I've never come across this before.
#10
It depends very much on your level of shader complexity, but there's a good amount of performance to be had in some cases.
06/04/2005 (4:14 pm)
Same reason you use LOD, use a "cheap" shader when it doesn't matter if that 2 pixel high unit in the distance is normal-mapped while reflecting a dynamic cube map etc. :)It depends very much on your level of shader complexity, but there's a good amount of performance to be had in some cases.
#11
You mean progressive shaders that drop stages as the distance from the camera increases? I didn't realise people called these LOD shaders as I associated that with mesh reduction algos. I'll file that term away in the greymatter archive.
06/05/2005 (3:14 am)
@Gareth You mean progressive shaders that drop stages as the distance from the camera increases? I didn't realise people called these LOD shaders as I associated that with mesh reduction algos. I'll file that term away in the greymatter archive.
#12
@Tim - Ummm.... at the moment not that i know of. You just have to look at shader code to figure out what it's doing. Maybe someone can get the shader generator to put some extra comments in the file to better describe the features the particular shader is supporting.
06/05/2005 (10:16 pm)
Yea shader LOD is gonna be needed eventually.@Tim - Ummm.... at the moment not that i know of. You just have to look at shader code to figure out what it's doing. Maybe someone can get the shader generator to put some extra comments in the file to better describe the features the particular shader is supporting.
#13
Delete all the shaderXxxx.hlsl files, and re run TSE you'll end up with 1 set of shaders. If you make any changes, tho they will be lost/overwritten the next time you run tse. So what you'll need to do is either comment out the #define in shadergen.cpp or just copy the shader set to some other file name and make a customMaterial out of it
hope that helps
:)
06/06/2005 (7:32 am)
@tim - You could just comment out ALL the addMaterialMapping() lines. except the one you want to generate.Delete all the shaderXxxx.hlsl files, and re run TSE you'll end up with 1 set of shaders. If you make any changes, tho they will be lost/overwritten the next time you run tse. So what you'll need to do is either comment out the #define in shadergen.cpp or just copy the shader set to some other file name and make a customMaterial out of it
hope that helps
:)
#14
Out of curiosity what shaders does everyone think should be procedural? I'm talking specifically, not generally.
Cheers -- Tim.
06/07/2005 (4:28 am)
Thanks Kevin and Tom - I'll look deeper into it. I'd like to see is the material name as part of the shaderXxxx file name or at the very least in the comments or code somewhere. No biggie, I just would have thought the material name would have been found easily on-hand in the context of when the procedural shader code is generated. If there is a registry or dictionary in TSE that contains the material name, I would appreciate it if someone could guide me down the right path.Out of curiosity what shaders does everyone think should be procedural? I'm talking specifically, not generally.
Cheers -- Tim.
#15
look in shadergen.cpp thats where the shader file is created, something like generateshader, and printheader i think. -dont have the code here to check :)
You could generate a comment by itterating through GFXShaderFeatureData enums it should have all the stuff your after..
good luck!
06/07/2005 (2:51 pm)
Hey tim, look in shadergen.cpp thats where the shader file is created, something like generateshader, and printheader i think. -dont have the code here to check :)
You could generate a comment by itterating through GFXShaderFeatureData enums it should have all the stuff your after..
good luck!
#16
06/07/2005 (2:53 pm)
Yea your comment in the shader will have to be a listing of the features the shader implements since one shader might be used by several different materials.
#17
06/07/2005 (3:03 pm)
Just took a quick look into the code, and looks like brian is way ahead of us :)from gfxStructs.h
struct GFXShaderFeatureData
{
...
// General feature data for a pass or for other purposes.
bool features[NumFeatures];
// This is to give hints to shader creation code. It contains
// all the features that are in a material stage instead of just
// the current pass.
bool materialFeatures[NumFeatures];
Torque Owner Brian Ramage
Default Studio Name
I'd also like to get bump-environment mapping in there, and refraction. They are both possible with custom materials, but could/should be moved to the procedural set.
Bloom is possible with the glow buffer. Not planning anything beyond that.
Wood will not be part of the procedural set. You are much better off using a photo-source texture unless you have a real toon look going on. Procedural wood usually looks like crap.