Shiny pictures and upcoming resources
by Alex Scarborough · 09/24/2006 (11:57 pm) · 23 comments
It's been nearly a month since my last .plan, so seems like a good time for another one. And what a month it's been. Actually, what a week it's been. Illumina released a public demo, which marks the first time my shader work has been available in any form to the general public. Some people liked it, others didn't. Many had outdated drivers. It's definitely a learning experience for everyone involved. On a more personal note, I can't play the Illumina demo because I don't have a PC. So, I have no clue what it really looks like. It'd be great if someone could send me some screenshots or something.
More recently, Alan James put up a .plan showing off some new screenshots of Code Name: Monster Island. To be perfectly honest, I thought that Alan had switched over to TSE. Between the insane detail on that sewer monster, and how stunning the TLK shadows look it just seemed like TSE. Definitely a reminder that while code can improve the look of a game, nothing beats good art.
But what have I been up to? Believe it or not, I have been primarily focussing on backend work. Things that aren't shiny. Things that don't involve rendering at all. Well, mostly. I've done some rendering work.
First of all, I recently acquired GPU Gems, ShaderX3, and ShaderX4 (IGC refund). They're all incredible books, and anyone with an interest in rendering with shaders should seriously consider purchasing them.
Of course, it's hard to have three shiny new books on shaderized rendering, and not try and implement at least one effect. I decided that sub surface scattering was sufficiently shiny, and required very minimal work to get a basic version of it running. Kork got a nice overhaul from
to
Combine that with spherical harmonic lighting...
And then for kicks, do some image post processing
Not bad for an hour or so of work. That sums up my shiny rendering work in the past month though. So, let's look at the backend work I've done. I swear it isn't (too) dull. If it is, just skip to the end of the .plan which discusses those upcoming resources promised in the title. Then come back here and read about the boring backend work because it is the upcoming resources.
Torque's GUI System
has issues. Let's look at this example of a reasonably nice GUI:
Notice that it is instantly obvious that the last scrollbar is inactive. That should be a very simple thing to do. And indeed, Torque's gui system does have a nice setting for an inactive color to use if the control is inactive.
At least, it would be a nice setting if any of the gui controls actually used it. Oh, sure, guiTextEditCtrl uses it somewhat, but that's really about it. guiSliderCtrl? Nope, not at all. That was actually rather irritating, and was also a five minute fix.
Look ma, it's inactive!
After some more tweaking, I was able to get the default gui rendering commands (that maybe 1/2 the guis use) to use the inactive color so long as they are told that the gui is inactive. It isn't complete, but it's a start.
So, here's a shot of the sample GUI above, slightly modified, in Torque.
guiMLTextCtrl
Oh man, where to start on this bad boy. The code is messy and buggy. Things like selecting text in the middle of a line simply don't work. It subtracts 64 from 64, multiplies by 3, adds 1, and wonders why it doesn't get 12. Again, most of the bugs prove to be a simple five minute fix.
How I feel after having to fix setting the cursor position.
With these little bugs fixed, I tackled a somewhat larger task. Something which has been attempted on a couple of occasions is syntax highlighting in GuiMLTextEditCtrl, most notably in the script editor resource. However, using TorqueML to do syntax highlighting is a bad bad thing.
So I wrote a script controlled system for doing syntax highlighting.
Yes that is GLSL. Yes the colors are a little odd. I lifted them directly from Apple's GLSLEditorSample. In the console, I'm about to tell that GuiMLTextCtrl that half3 should be recognized as a GLSLDatatype keyword. No, half3 is not a GLSL datatype. Welcome to the joys of optimizing GLSL for nVidia cards (see those messy #defines at the top of the file? They are the happy dance GLSL developers must go through to make GLSL not suck on the FX series).
Dynamic Range Lighting
There are now, what, four dynamic range lighting resources for TGE? Three of which I've either written or include my code. All of them have one major caveat: they all force a sync point between CPU and GPU. This sync point kills performance, and can cause some decidedly unpleasant issues. TSE's DRL implementation avoids the sync point by not making the DRL effect dependent on any previous frame, which has some downsides of its own.
But hey, we have shaders now. Textures can hold whatever kind of data we want. Why should they be limited to lookup tables and color information?
So, I moved DRL over entirely to the GPU (if it has access to shaders at least). No more sync point. It determines the new tonemap in a shader, writes the scale and bias into a 1x1 texture, and then at every pixel during DRL rendering it grabs the scale and bias from that texture and applies it. This 1) brought a 20-50% FPS boost, which is just plain awesome, and 2) makes it very easy to adjust the DRL effect as a whole. All the CPU does is bind the various shaders and tell the GPU to downsample some textures. Shaders handle everything else.
How does it track data across multiple frames? A cheap hack. It has two 1x1 textures, and simply alternates between them. When writing data to one 1x1 texture, the other is bound and the current DRL info is read from it, modified, then written into the texture.
As a result of this, something happened that I haven't seen in a long long time. Torque became CPU bound again. Time spent in OpenGL went from 60-70% to 30-40%. Surely though, that number can be improved...
glBegin
is the devil. Here's a shot from the OpenGL Profiler, showing the top 3 most time consuming OpenGL calls in Torque.
Look at #1. Look at just how much more time it takes over #2. Look at how each call is, on average, 150 more expensive than #2.
I've begun a quest to completely purge glBegin from Torque. Does anyone want to help me?
TGE Shader Demo
I've been planning to release a demo of my shader work to the general public for some time now. The original plan was to have it ready for IGC. When IGC was cancelled, the demo was backburnered for a while. It's slowly coming back up on the list of things to do though. In fact, you probably noticed that my example gui was a shader parameter editor, and that my example of syntax highlighting was GLSL.
I've dropped the idea of a simple demo to show off what I can do. That's dull, and nothing more than an ego booster. Instead, I'd prefer to see what everyone else can do. Instead of releasing a demo showing off my work, I'm now planning to release a Torque based GLSL shader editor, allowing people to write and test their own GLSL shaders in a real game environment. Furthermore, it will be crossplatform unlike FXComposer or RenderMonkey, allowing Mac and Linux users to get in on the fun as well. It won't be as full featured as FXComposer or RenderMonkey, as the goal is to provide a sandbox, and not a serious shader IDE.
At the moment, I have around 80 example shaders, in addition to four libraries of shader functions, including image manipulation and lighting effects, which GLSL shaders may be linked against. I have also integrated 3Dlabs' GLSLParserTest into Torque, allowing people to check how conformant the GLSL implementation of their driver is with one simple console function.
You know what? Just supporting GLSL is boring. The TGE Shader Demo also currently supports Cg, ARB_vertex_program/ARB_fragment_program, and ATI_text_fragment shader. Admittedly, a fair amount of work still needs to be done to get those up to the same caliber as the GLSL implementation, but it is capable of loading and using shaders in those languages.
But that's all down the line a bit.
Upcoming resources
The gui work I've done will be packaged up into a resource after TGE 1.4.2 is officially released. It's quite handy. Jeff "Reno" Raab is also currently working on an integrated script editor using the syntax highlighting code I've written, so that should pop up within a month or so as well.
In my quest to purge glBegin from Torque, I've rewritten the particle and decal renders to batch rendering (and in the process broke the TLK enhancements to decals and particle emitters). Resources for those in stock TGE will probably go up after TGE 1.4.2 is officially released, with TLK resources shortly afterwards.
I will probably also release a modified DRL resource which purges glBegin. glBegin isn't exactly a major issue when DRL is running, but every little bit helps.
While I'm at all this batching, I should probably fix the precipitation renderer. Another after 1.4.2 is out resource.
And then there's the TGE Shader Demo. Ideally, I'll have that out before then end of this year (maybe later if I have to buy a commercial license to release it since it isn't a game). I'm trying to make it as easy to use as possible, which will probably necessitate a couple more editors, various features, and a lot of gui work. I also need to triple check the 3Dlabs license, seeing as a pretty large chunk of the shader code used in the TGE Shader Demo came straight from various 3Dlabs samples and utilities.
Well, that sums up my month. Or at least what I did with my free time in the past month.
If anyone wants to try out the gui fixes/enhancements or batched renderers before I put them up as resources, just email me and I'll be more than happy to send you what code I have right now. I really could use some feedback on how useable the syntax highlighting is, and if the batched renderers bring any sort of significant performance boost.
More recently, Alan James put up a .plan showing off some new screenshots of Code Name: Monster Island. To be perfectly honest, I thought that Alan had switched over to TSE. Between the insane detail on that sewer monster, and how stunning the TLK shadows look it just seemed like TSE. Definitely a reminder that while code can improve the look of a game, nothing beats good art.
But what have I been up to? Believe it or not, I have been primarily focussing on backend work. Things that aren't shiny. Things that don't involve rendering at all. Well, mostly. I've done some rendering work.
First of all, I recently acquired GPU Gems, ShaderX3, and ShaderX4 (IGC refund). They're all incredible books, and anyone with an interest in rendering with shaders should seriously consider purchasing them.
Of course, it's hard to have three shiny new books on shaderized rendering, and not try and implement at least one effect. I decided that sub surface scattering was sufficiently shiny, and required very minimal work to get a basic version of it running. Kork got a nice overhaul from
to
Combine that with spherical harmonic lighting...
And then for kicks, do some image post processing
Not bad for an hour or so of work. That sums up my shiny rendering work in the past month though. So, let's look at the backend work I've done. I swear it isn't (too) dull. If it is, just skip to the end of the .plan which discusses those upcoming resources promised in the title. Then come back here and read about the boring backend work because it is the upcoming resources.
Torque's GUI System
has issues. Let's look at this example of a reasonably nice GUI:
Notice that it is instantly obvious that the last scrollbar is inactive. That should be a very simple thing to do. And indeed, Torque's gui system does have a nice setting for an inactive color to use if the control is inactive.
At least, it would be a nice setting if any of the gui controls actually used it. Oh, sure, guiTextEditCtrl uses it somewhat, but that's really about it. guiSliderCtrl? Nope, not at all. That was actually rather irritating, and was also a five minute fix.
Look ma, it's inactive!After some more tweaking, I was able to get the default gui rendering commands (that maybe 1/2 the guis use) to use the inactive color so long as they are told that the gui is inactive. It isn't complete, but it's a start.
So, here's a shot of the sample GUI above, slightly modified, in Torque.
guiMLTextCtrl
Oh man, where to start on this bad boy. The code is messy and buggy. Things like selecting text in the middle of a line simply don't work. It subtracts 64 from 64, multiplies by 3, adds 1, and wonders why it doesn't get 12. Again, most of the bugs prove to be a simple five minute fix.
How I feel after having to fix setting the cursor position.With these little bugs fixed, I tackled a somewhat larger task. Something which has been attempted on a couple of occasions is syntax highlighting in GuiMLTextEditCtrl, most notably in the script editor resource. However, using TorqueML to do syntax highlighting is a bad bad thing.
So I wrote a script controlled system for doing syntax highlighting.
Yes that is GLSL. Yes the colors are a little odd. I lifted them directly from Apple's GLSLEditorSample. In the console, I'm about to tell that GuiMLTextCtrl that half3 should be recognized as a GLSLDatatype keyword. No, half3 is not a GLSL datatype. Welcome to the joys of optimizing GLSL for nVidia cards (see those messy #defines at the top of the file? They are the happy dance GLSL developers must go through to make GLSL not suck on the FX series).
Dynamic Range Lighting
There are now, what, four dynamic range lighting resources for TGE? Three of which I've either written or include my code. All of them have one major caveat: they all force a sync point between CPU and GPU. This sync point kills performance, and can cause some decidedly unpleasant issues. TSE's DRL implementation avoids the sync point by not making the DRL effect dependent on any previous frame, which has some downsides of its own.
But hey, we have shaders now. Textures can hold whatever kind of data we want. Why should they be limited to lookup tables and color information?
So, I moved DRL over entirely to the GPU (if it has access to shaders at least). No more sync point. It determines the new tonemap in a shader, writes the scale and bias into a 1x1 texture, and then at every pixel during DRL rendering it grabs the scale and bias from that texture and applies it. This 1) brought a 20-50% FPS boost, which is just plain awesome, and 2) makes it very easy to adjust the DRL effect as a whole. All the CPU does is bind the various shaders and tell the GPU to downsample some textures. Shaders handle everything else.
How does it track data across multiple frames? A cheap hack. It has two 1x1 textures, and simply alternates between them. When writing data to one 1x1 texture, the other is bound and the current DRL info is read from it, modified, then written into the texture.
As a result of this, something happened that I haven't seen in a long long time. Torque became CPU bound again. Time spent in OpenGL went from 60-70% to 30-40%. Surely though, that number can be improved...
glBegin
is the devil. Here's a shot from the OpenGL Profiler, showing the top 3 most time consuming OpenGL calls in Torque.
Look at #1. Look at just how much more time it takes over #2. Look at how each call is, on average, 150 more expensive than #2.
I've begun a quest to completely purge glBegin from Torque. Does anyone want to help me?
TGE Shader Demo
I've been planning to release a demo of my shader work to the general public for some time now. The original plan was to have it ready for IGC. When IGC was cancelled, the demo was backburnered for a while. It's slowly coming back up on the list of things to do though. In fact, you probably noticed that my example gui was a shader parameter editor, and that my example of syntax highlighting was GLSL.
I've dropped the idea of a simple demo to show off what I can do. That's dull, and nothing more than an ego booster. Instead, I'd prefer to see what everyone else can do. Instead of releasing a demo showing off my work, I'm now planning to release a Torque based GLSL shader editor, allowing people to write and test their own GLSL shaders in a real game environment. Furthermore, it will be crossplatform unlike FXComposer or RenderMonkey, allowing Mac and Linux users to get in on the fun as well. It won't be as full featured as FXComposer or RenderMonkey, as the goal is to provide a sandbox, and not a serious shader IDE.
At the moment, I have around 80 example shaders, in addition to four libraries of shader functions, including image manipulation and lighting effects, which GLSL shaders may be linked against. I have also integrated 3Dlabs' GLSLParserTest into Torque, allowing people to check how conformant the GLSL implementation of their driver is with one simple console function.
You know what? Just supporting GLSL is boring. The TGE Shader Demo also currently supports Cg, ARB_vertex_program/ARB_fragment_program, and ATI_text_fragment shader. Admittedly, a fair amount of work still needs to be done to get those up to the same caliber as the GLSL implementation, but it is capable of loading and using shaders in those languages.
But that's all down the line a bit.
Upcoming resources
The gui work I've done will be packaged up into a resource after TGE 1.4.2 is officially released. It's quite handy. Jeff "Reno" Raab is also currently working on an integrated script editor using the syntax highlighting code I've written, so that should pop up within a month or so as well.
In my quest to purge glBegin from Torque, I've rewritten the particle and decal renders to batch rendering (and in the process broke the TLK enhancements to decals and particle emitters). Resources for those in stock TGE will probably go up after TGE 1.4.2 is officially released, with TLK resources shortly afterwards.
I will probably also release a modified DRL resource which purges glBegin. glBegin isn't exactly a major issue when DRL is running, but every little bit helps.
While I'm at all this batching, I should probably fix the precipitation renderer. Another after 1.4.2 is out resource.
And then there's the TGE Shader Demo. Ideally, I'll have that out before then end of this year (maybe later if I have to buy a commercial license to release it since it isn't a game). I'm trying to make it as easy to use as possible, which will probably necessitate a couple more editors, various features, and a lot of gui work. I also need to triple check the 3Dlabs license, seeing as a pretty large chunk of the shader code used in the TGE Shader Demo came straight from various 3Dlabs samples and utilities.
Well, that sums up my month. Or at least what I did with my free time in the past month.
If anyone wants to try out the gui fixes/enhancements or batched renderers before I put them up as resources, just email me and I'll be more than happy to send you what code I have right now. I really could use some feedback on how useable the syntax highlighting is, and if the batched renderers bring any sort of significant performance boost.
About the author
#2
f.pischedda AT bigbangsol DOT com and I'll be happy to test it and return results to you (I'm interested in GUI fixes also).
Bye
09/25/2006 (12:31 am)
Hi, I'm really interested in your work; I'd like to test your batch rendering code as I'm experiencing some slow downs due to high poly count in the scene; if you like some feedback send me your code at f.pischedda AT bigbangsol DOT com and I'll be happy to test it and return results to you (I'm interested in GUI fixes also).
Bye
#4
09/25/2006 (12:38 am)
Awesome work. While this is over my head it is interesting! Keep it up 
#5
I didn't even knew that you can purge some glBegin calls... that's really interesting.
09/25/2006 (1:46 am)
Alex, you ROCK.I didn't even knew that you can purge some glBegin calls... that's really interesting.
#6
09/25/2006 (4:55 am)
Wow, you've been keeping busy. I'm sure the Mac developers will love you.
#7
09/25/2006 (5:53 am)
Nice work alex. i see you been looking at the CG DRL res, then decided to update yours. good job :), now i dont know what res i want to use :S. BTW alex, whats fps difference between this res and your older res?
#8
You asked for an illumina screenshot or something that displays your effects. I'm linking a short video of the water refraction in action. Video: 7 seconds, 1024x768 resolution, AVI compressed with xVid, under 5 megs file size.
www.aaronellis.net/illuminaALEX.avi
If this works for you, I might be able to throw some more together.
09/25/2006 (6:15 am)
@AlexYou asked for an illumina screenshot or something that displays your effects. I'm linking a short video of the water refraction in action. Video: 7 seconds, 1024x768 resolution, AVI compressed with xVid, under 5 megs file size.
www.aaronellis.net/illuminaALEX.avi
If this works for you, I might be able to throw some more together.
#9
09/25/2006 (6:45 am)
T Squared, the "CG DRL" resources are kind of..fake. Yes, that's a good word. Alex is doing more advanced things such as sub surface scattering, spherical harmonic lighting, and I think he has a blur shader thrown in.
#10
09/25/2006 (6:48 am)
Now I need to try even harder to learn shader implementation (Darn you C# taking up all my time! RWWAARR!)
#11
--clint
09/25/2006 (6:59 am)
Alex, great work on the gui updates! I know everyone else likes the shiny shader pictures, but it's actually the GUI control updates that impress me. That's the sort of polish that takes effort to dig down and track down all of the easy-yet-annoying-to-find-bugs-that-should-have-been-done-right-in-the-first-place-anyways. Just wanted to say, definite props from me! I look forward to these changes getting merged into HEAD.--clint
#12
@T Squared: There will be no GLSL DRL resource. Again, I have been asked not to publicly release my GLSL code. That section was more of a nice example of an alternate use for textures to help accelerate rendering. Besides, I shaderized DRL months before the Cg resource came out. Check out my plan entitled "Because I can't just drop DRL..."
@Aaron: Thanks for the video. That black reflection is ugly. I wonder if it's the reflection or a messed up specular highlight.
@Matt: The DRL bloom uses a blur shader, though I need to go in and spiff it up a bit.
@Clint: The bugs weren't too hard to find. Did you think I was kidding when I said "It subtracts 64 from 64, multiplies by 3, adds 1, and wonders why it doesn't get 12"? That sums up what was going on in the cursor position code right there.
09/25/2006 (7:48 am)
@Mathieu: Replacing glBegin involves moving the data into an array, and then drawing it with glDrawArrays or glDrawElements (preferably glDrawElements). That's all there is to it. But, if you're already replacing render code, it's also a perfect time to batch together things so instead of 200 calls to glBegin, you might have one call to glDrawElements (more or less how the batched particle renderer winds up working).@T Squared: There will be no GLSL DRL resource. Again, I have been asked not to publicly release my GLSL code. That section was more of a nice example of an alternate use for textures to help accelerate rendering. Besides, I shaderized DRL months before the Cg resource came out. Check out my plan entitled "Because I can't just drop DRL..."
@Aaron: Thanks for the video. That black reflection is ugly. I wonder if it's the reflection or a messed up specular highlight.
@Matt: The DRL bloom uses a blur shader, though I need to go in and spiff it up a bit.
@Clint: The bugs weren't too hard to find. Did you think I was kidding when I said "It subtracts 64 from 64, multiplies by 3, adds 1, and wonders why it doesn't get 12"? That sums up what was going on in the cursor position code right there.
#13
I never ask for the GLSL DRL. i just wonder when you where planning on releasing this new shiny resource. I'll go check out your more updated DRL res, and see how it is. BTW, did alan use your DRL res for his game?
09/25/2006 (8:53 am)
@AlexI never ask for the GLSL DRL. i just wonder when you where planning on releasing this new shiny resource. I'll go check out your more updated DRL res, and see how it is. BTW, did alan use your DRL res for his game?
#15
This is great for us Mac developers.
09/25/2006 (9:19 am)
Wow. Can't really think of much more to say than that.This is great for us Mac developers.
#16
Yeah, the black wave crests are ugly. I ran into something similar in my project, and eventually was able to eliminate it. I can't remember what I did, but I'll look into it.
The refractivity looks great, but seems to do weird things to particles with transparency. I haven't tested it much, so it could be something else. Here's a vid of the particle wierdness in action . . .
www.aaronellis.net/illuminaALEX2.avi
Note: The blocky artifacts around the particles are not a result of compression, since they are visible in the actual game.
09/25/2006 (9:42 am)
@AlexYeah, the black wave crests are ugly. I ran into something similar in my project, and eventually was able to eliminate it. I can't remember what I did, but I'll look into it.
The refractivity looks great, but seems to do weird things to particles with transparency. I haven't tested it much, so it could be something else. Here's a vid of the particle wierdness in action . . .
www.aaronellis.net/illuminaALEX2.avi
Note: The blocky artifacts around the particles are not a result of compression, since they are visible in the actual game.
#17
Also, Monster Island is currently using GLSL DRL.
@Tim: Yes, yes it is. Thankfully, it's all more or less related so I'm still only dealing with one code base.
@Rubes: Macs need OpenGL dev tools too. It's actually rather confusing that windows, which has Dx and OpenGL has all the OpenGL tools, and Mac/Linux, which only have OpenGL, have no OpenGL tools.
@Aaron: I remember that bug! I must never have given them the fix... basically, my first pass on refractive water was horrific. The water is actually rendered *AFTER* everything else, and handles blending in the shader. The transparent particles completely throw off it's blending. See, this is a very stupid way to approach refraction. I have sinced changed it to work properly i.e. water renders at normal time.
09/25/2006 (2:00 pm)
@T Squared: I'll release a DRL resource that purges glBegin within the next week or so, give or take. Depends on how much time I have to put into it.Also, Monster Island is currently using GLSL DRL.
@Tim: Yes, yes it is. Thankfully, it's all more or less related so I'm still only dealing with one code base.
@Rubes: Macs need OpenGL dev tools too. It's actually rather confusing that windows, which has Dx and OpenGL has all the OpenGL tools, and Mac/Linux, which only have OpenGL, have no OpenGL tools.
@Aaron: I remember that bug! I must never have given them the fix... basically, my first pass on refractive water was horrific. The water is actually rendered *AFTER* everything else, and handles blending in the shader. The transparent particles completely throw off it's blending. See, this is a very stupid way to approach refraction. I have sinced changed it to work properly i.e. water renders at normal time.
#18
09/25/2006 (2:54 pm)
Looking good Alex! I look forward to your "sandbox." It should prove fun to play around in.
#19
09/25/2006 (2:58 pm)
@Nick: It's a ton of fun to play around in. I lost a good half hour of time when I finished the parameter editor, just pushing around sliders and selecting random colors to see what happened when I did things like set the index of refraction for the water to yellow, or the sub surface scattering wrap to turquoise.
#20
You should really try getting bumpmap/normalmaps working on dts next. That would be awesome. of course after you finish this one off. There is already a dts bumpmap res out there, but as far as i can see its a bit outdated. or has used an earlier version of TGE. that i cant compare the code with. so no luck on implementing that one.
09/25/2006 (4:01 pm)
@AlexYou should really try getting bumpmap/normalmaps working on dts next. That would be awesome. of course after you finish this one off. There is already a dts bumpmap res out there, but as far as i can see its a bit outdated. or has used an earlier version of TGE. that i cant compare the code with. so no luck on implementing that one.
Torque Owner Jeff Raab
[ghc]games
Awsome work del, now get me that shader editor so i can jack around with stuff already!