TSE - some more info
by Brian Ramage · in Torque Game Engine · 03/20/2004 (8:36 pm) · 72 replies
Hey guys, I've seen some questions asked about TSE in a few threads, I thought I'd answer a few here and clear some things up.
The TSE is a full engine, pretty much up to date with TGE 1.2. Merging a project from 1.2 could be a bit of a pain as we have changed every line of code that was previously pure OpenGL - so a lot of files have been at least slightly modified. In addition, there are a lot of features that have not yet been converted over to the new GFX layer including some editor functionality.
The TSE WILL fall back from pixel shader 2.0 to 1.0 all the way back to fixed function, so it will work on all types of hardware. There're two ways to specify shaders - through the base Material interface, and through the CustomMaterial interface.
The base Materials take datablock input for the textures, cubemaps, bumpmaps, specular settings, glow, etc and procedurally turn them into shaders. It even has multiple stages so you can create multipass materials. You don't need to write a line of code for those shaders - Joe and Weston used the base Materials to do all the shaders on the orc in the demo.
The second way is to use the CustomMaterial interface. With that, you can write your own HLSL or assembly shaders and specify the whole chain of fallbacks for each type of shader hardware. That means you can pretty much take some examples from RenderMonkey or FX Composer and drop them into the TSE and watch them run.
The graphics layer is a layer that we implemented that has its own API - a sort of cross between OpenGL and D3D. OpenGL and D3D implementations are derived from the GFX layer and take the same commands from it. This allows complete separation from a specific graphics API. We focused on D3D support from the beginning because it has more mature shader technology. We will be working on the OpenGL side of it soon, but currently it is not supported. Since we have so many Mac and Linux users, obviously we aren't going to ignore this, it is a priority.
The TSE is a full engine, pretty much up to date with TGE 1.2. Merging a project from 1.2 could be a bit of a pain as we have changed every line of code that was previously pure OpenGL - so a lot of files have been at least slightly modified. In addition, there are a lot of features that have not yet been converted over to the new GFX layer including some editor functionality.
The TSE WILL fall back from pixel shader 2.0 to 1.0 all the way back to fixed function, so it will work on all types of hardware. There're two ways to specify shaders - through the base Material interface, and through the CustomMaterial interface.
The base Materials take datablock input for the textures, cubemaps, bumpmaps, specular settings, glow, etc and procedurally turn them into shaders. It even has multiple stages so you can create multipass materials. You don't need to write a line of code for those shaders - Joe and Weston used the base Materials to do all the shaders on the orc in the demo.
The second way is to use the CustomMaterial interface. With that, you can write your own HLSL or assembly shaders and specify the whole chain of fallbacks for each type of shader hardware. That means you can pretty much take some examples from RenderMonkey or FX Composer and drop them into the TSE and watch them run.
The graphics layer is a layer that we implemented that has its own API - a sort of cross between OpenGL and D3D. OpenGL and D3D implementations are derived from the GFX layer and take the same commands from it. This allows complete separation from a specific graphics API. We focused on D3D support from the beginning because it has more mature shader technology. We will be working on the OpenGL side of it soon, but currently it is not supported. Since we have so many Mac and Linux users, obviously we aren't going to ignore this, it is a priority.
#2
03/20/2004 (8:53 pm)
Thanks for the details, I'll make sure not to change any render code for now just to be on the safe side. I've been adding resources and getting them working in 1.2.x, but I think I'll concentrate on modeling until this comes out.
#3
That concerns me because I want to know if I should start a project now, and port it to TSE at a later development stage, or just wait for Torque2 to start it.
The point is, if TSE is going to have all the features from Torque 1.2, being easy to port, I'll start the development now. If things will change radically, I'll wait.
Thanks for the clarification!
03/20/2004 (9:41 pm)
Is the TSE what, at some old screenshot from GG, was entitled "Torque 2", or will there still be a Torque 2 on top of TSE?That concerns me because I want to know if I should start a project now, and port it to TSE at a later development stage, or just wait for Torque2 to start it.
The point is, if TSE is going to have all the features from Torque 1.2, being easy to port, I'll start the development now. If things will change radically, I'll wait.
Thanks for the clarification!
#4
I think the EA will be out in not more than three months. Possibly a lot less, but don't quote me on that.
As for development - start with TGE1.2. The early adopter releases are NOT for the faint of heart, NOT for people who want to release a serious product with them, and DEFINITELY not for people who aren't comfortable with C++ and shader technology. We will be doing serious development work on it from now until the time TSE 1.0 is released, and things will be broken, in flux, or just plain wrong at any given moment in time.
That said, if you want to play with the very latest in graphics tech, this is a rare and possibly invaluable opportunity. We'd love to have you get your hands dirty with it. But be aware that it's not in a final state, and that we won't be able to hold your hand with it.
Hopefully that makes things clear - this is a rant sort of for anyone who wants to buy TSE, not just you in particular. ;)
Best plan if you want to make a game: Work on TGE1.2, get the game done, port to TSE at your leisure.
03/21/2004 (12:29 am)
RSN (tm).I think the EA will be out in not more than three months. Possibly a lot less, but don't quote me on that.
As for development - start with TGE1.2. The early adopter releases are NOT for the faint of heart, NOT for people who want to release a serious product with them, and DEFINITELY not for people who aren't comfortable with C++ and shader technology. We will be doing serious development work on it from now until the time TSE 1.0 is released, and things will be broken, in flux, or just plain wrong at any given moment in time.
That said, if you want to play with the very latest in graphics tech, this is a rare and possibly invaluable opportunity. We'd love to have you get your hands dirty with it. But be aware that it's not in a final state, and that we won't be able to hold your hand with it.
Hopefully that makes things clear - this is a rant sort of for anyone who wants to buy TSE, not just you in particular. ;)
Best plan if you want to make a game: Work on TGE1.2, get the game done, port to TSE at your leisure.
#5
03/21/2004 (1:28 am)
Mmmmm... I cant wait to get my hands on the tech. Hopefully it'll teach me as much about shaders as Torque has done about everything else :D Will it have any documentation straight out of the 'box'? I doubt it, aside from code comments, just thought I'd ask ;) 
#6
03/21/2004 (8:01 am)
The API is fully documented. I'm sure other stuff will be too. It is always the intention to improve documentation and since we use DOxygen, it's very easy to document as you write it. Just look at TNL! That is a great example of documentation improvement. I don't know about the other sections, but if I have my way, it will all be documented so that no-one ever has to say, "What the heck is TraverseColor?"
#7
03/21/2004 (8:35 am)
Hehe, Pat, but then why did you leave those offending lines as comments in the code ;) 
#8
03/21/2004 (8:47 am)
@Ben: Thanks for the info. It does make things clear for me at least. I will be watching to see how it progress in the future and see if it will suite my needs. Oh yeah...will there be documentation with this or is it still being made?
#9
03/21/2004 (9:06 am)
What default shaders are planned? Is now a good time to start voting for Most Wanted shaders?
#10
03/21/2004 (9:10 am)
I blame the Traverse Muskrat for those comments.
#11
I want to first commend you and the others involved in the work on the shaders. I do have a few questions. Our project has been working with our own implementation of OpenGL pixel shaders and dynamic lighting in Torque for some time but would like to take advantage of what will be available from TSE. I am hoping you can provide some feedback on the following questions:
1) You mention configuring shaders using datablocks which I assume refers to scripting. Will it also be accessible through some in-game editor similar to the mission and gui editors?
2) Will the show tool allow artists to prototype and render shaders in real-time?
3) Does the TSE support artist-generated and procedurally generated bumpmaps and specular maps? If so, will there be a tool to convert greyscale bumpmap textures to normal maps?
4) If TSE supports artist-generated bump and spec maps, what is the texture size limit?
5) Is S3TC compression supported (.DDS) for textures?
6) Is shader information exported with shapes through the DTS and DIF exporters? If so, will GarageGames be responsible for updating and supporting the exporters with third-party modeling and mapping tools?
7) Can the shaders be toggled on/off dynamically in-game?
8) Is there shader support and dynamic per-pixel lighting for all surfaces including interiors, waterblocks, and terrain?
9) What is the recommended video card to access all shader capabilities?
10) Will it be possible to use shaders with the particle engine?
11) When will OpenGL support be added?
12) Will there be a demo?
03/21/2004 (9:41 am)
Brian,I want to first commend you and the others involved in the work on the shaders. I do have a few questions. Our project has been working with our own implementation of OpenGL pixel shaders and dynamic lighting in Torque for some time but would like to take advantage of what will be available from TSE. I am hoping you can provide some feedback on the following questions:
1) You mention configuring shaders using datablocks which I assume refers to scripting. Will it also be accessible through some in-game editor similar to the mission and gui editors?
2) Will the show tool allow artists to prototype and render shaders in real-time?
3) Does the TSE support artist-generated and procedurally generated bumpmaps and specular maps? If so, will there be a tool to convert greyscale bumpmap textures to normal maps?
4) If TSE supports artist-generated bump and spec maps, what is the texture size limit?
5) Is S3TC compression supported (.DDS) for textures?
6) Is shader information exported with shapes through the DTS and DIF exporters? If so, will GarageGames be responsible for updating and supporting the exporters with third-party modeling and mapping tools?
7) Can the shaders be toggled on/off dynamically in-game?
8) Is there shader support and dynamic per-pixel lighting for all surfaces including interiors, waterblocks, and terrain?
9) What is the recommended video card to access all shader capabilities?
10) Will it be possible to use shaders with the particle engine?
11) When will OpenGL support be added?
12) Will there be a demo?
#12
2) Like I said - no realtime editor. The show tool does have shader support, though.
3) We have been using the photoshop plugin that Nvidia released. All our art has been artist-created thus far. It loads standard image files; no reason you couldn't just blast data to a GBitmap and go on your merry way, though.
4) How much video RAM do you have?
5) If you want to write a loader, sure. It's not likely to be officially supported, as DDS isn't a crossplatform format.
6) See above. Materials are mapped by texture name.
7) What would the off-state do?
8) Interiors, yes. The terrain renderer is in a very excitingly broken state right now, so we haven't done a lot of work on fancy lighting. Same for water.
9) Any DX9 compatible card? I use a GeForce 5950 FX. Brian was doing work on an Radeon 9800 until he broke it. :P
10) I don't see why not. The particle engine is also in a state of borkitude right now.
11) When someone writes it.
12) At some point.
Remember, this is still an Early Adopter's release. We're many months from a final stable product. Many, many things are subject to change. Including what we do or don't, will or won't support.
-- Ben, Speaking for Brian
03/21/2004 (11:26 am)
1) Currently, you simply bind materials to texture names. So the material setup is done in the datablock and the material assignation is done at asset creation time (though you can change the mapping later). An editor would be really nice.2) Like I said - no realtime editor. The show tool does have shader support, though.
3) We have been using the photoshop plugin that Nvidia released. All our art has been artist-created thus far. It loads standard image files; no reason you couldn't just blast data to a GBitmap and go on your merry way, though.
4) How much video RAM do you have?
5) If you want to write a loader, sure. It's not likely to be officially supported, as DDS isn't a crossplatform format.
6) See above. Materials are mapped by texture name.
7) What would the off-state do?
8) Interiors, yes. The terrain renderer is in a very excitingly broken state right now, so we haven't done a lot of work on fancy lighting. Same for water.
9) Any DX9 compatible card? I use a GeForce 5950 FX. Brian was doing work on an Radeon 9800 until he broke it. :P
10) I don't see why not. The particle engine is also in a state of borkitude right now.
11) When someone writes it.
12) At some point.
Remember, this is still an Early Adopter's release. We're many months from a final stable product. Many, many things are subject to change. Including what we do or don't, will or won't support.
-- Ben, Speaking for Brian
#13
Can I asume that I will recieve the 1.0 release if I go with the EA program ??
03/21/2004 (11:42 am)
Hmm, as it seem, the EA release will be priced at $150 and the 1.0 release will be at $295/$995.Can I asume that I will recieve the 1.0 release if I go with the EA program ??
#14
--Carsten
Disclaimer: This is not about saving money ;) I wouldn't mind buying both TNL and TSE if I can save myself the time to implement the security layer myself. $200-$300 is about a day of work and I'm sure couldn't implement it in less.
03/21/2004 (11:49 am)
Will any of the enhancements made to the network layer in TNL flow back into the TSE codeline? I would particularly be interested in the security enhancements that have been made according to Mark's .plan, would save me a ton of work since I will need a fairly tamper proof communication layer between clients and the server.--Carsten
Disclaimer: This is not about saving money ;) I wouldn't mind buying both TNL and TSE if I can save myself the time to implement the security layer myself. $200-$300 is about a day of work and I'm sure couldn't implement it in less.
#15
03/21/2004 (11:57 am)
Quote:7) What would the off-state do?Giving the users the option to turn it off allows them to increase performance. I dont' think i'd want to put shaders into my game if I can't allow the users to turn them off.
#17
Just about everything CAN support shaders including particles and even GUI controls. We haven't had the bandwidth to get to those yet, but it will happen.
Currently datablocks are used for material definitions, but in the future the script language will change a bit and the materials will be defined more like Quake 3 shaders. Even further in the future, exporters will convert MAX or Maya materials into TSE materials so no scripting will be necessary. Scripting will always give you more options and control over the materials though.
Material shaders are created procedurally, so I guess the number of base shaders supported would be the combination of the following effects (off the top of my head) base texture, bumpmap, specular, cubemapping, fog, texture uv animation, detail mapping, and various other more subtle things like emissive textures, indoor light-normal maps, etc. It comes out to more than 1024 possible shaders, maybe even up to 4096, I'd have to check.
03/21/2004 (12:18 pm)
Ben pretty much fielded that correctly. On texture compression (S3TC, DXTC, .DDS, etc) it actually IS cross platform and will run on both OpenGL and D3D, so I don't see any problems with supporting that (knock on wood).Just about everything CAN support shaders including particles and even GUI controls. We haven't had the bandwidth to get to those yet, but it will happen.
Currently datablocks are used for material definitions, but in the future the script language will change a bit and the materials will be defined more like Quake 3 shaders. Even further in the future, exporters will convert MAX or Maya materials into TSE materials so no scripting will be necessary. Scripting will always give you more options and control over the materials though.
Material shaders are created procedurally, so I guess the number of base shaders supported would be the combination of the following effects (off the top of my head) base texture, bumpmap, specular, cubemapping, fog, texture uv animation, detail mapping, and various other more subtle things like emissive textures, indoor light-normal maps, etc. It comes out to more than 1024 possible shaders, maybe even up to 4096, I'd have to check.
#18
03/21/2004 (12:23 pm)
John - this would be handled by the fallback system. Eventually, yes, people will be able to choose what detail level they run at. Right now we're just getting the darn thing working. :)
#19
The user could set the level of shaders that the TSE will run, so you could set the level of shaders to ps 3.0, 2.0, or 1.0, or fall back completely to fixed function. Right now it detects what level of hardware you support and runs at the maximum level, but it would be easy to change the code to force it lower.
03/21/2004 (12:26 pm)
@Paul & John:The user could set the level of shaders that the TSE will run, so you could set the level of shaders to ps 3.0, 2.0, or 1.0, or fall back completely to fixed function. Right now it detects what level of hardware you support and runs at the maximum level, but it would be easy to change the code to force it lower.
#20
And sure I understand right now its just get the things working :)
03/21/2004 (12:52 pm)
Yeah I understand it can fall back if hardware doesn't support it. I'm just saying it would be nice to allow a setting to force it to a lower setting, like Brian mentions. I know I personally have turned shaders off in games before. My card supports them but the rest of my hardware really doesn't :pAnd sure I understand right now its just get the things working :)
Torque Owner Eric Petersen