Spritesheets and Video Memory in TGB
by Michael Woerister · 02/13/2007 (5:28 am) · 20 comments
2D games can take up quite a lot of video memory. They often feature smooth animations that need a lot of single frames and you constantly hear people complaining about TGB's high memory consumption. Fact is though that quite a significant percentage of video memory is wasted for storing completely transparent areas. For example take a look at this clipping of a regular sprite sheet (from Konami's 1993 'Batman Returns' on the SNES):
Only the green areas of the image carry relevant picture data whereas the red areas could be omitted without any loss. If the sprite sheet is stored as a PNG file those red areas won't make a real difference on your hard drive as they will compress practically to zero. However in video memory such compression is not available. Even if texture compression is available it is neither lossless nor is it very strong (50% gain at the best). So in video memory it does matter whether those red areas are stored or not.
Last Sunday I found the time to do some tinkering. TGB's image map system already is pretty nifty and if you hook into the place it is just a minimal code addition that has the task to find out the relevant area of every frame and then store its size and position relative the original size and position. The rendering code can then use this information to simulate the transparent areas of the image by just offsetting the smaller green area in the sprite. It took less than 2 hours to get a prototype working and the result is quite pleasing :)
BEFORE
AFTER
As you can see the sprite sheet created without this technique was sized 1024x256 pixels whereas the second sheet is sized only 512x128. That means it uses only a fourth of the memory the other sheet uses without losing any information.
And that gain can be achieved without any intervention on the side of game developer. Everything happens behind the scenes.
I think this would be quite a nice addition to TGB's image map system.
Only the green areas of the image carry relevant picture data whereas the red areas could be omitted without any loss. If the sprite sheet is stored as a PNG file those red areas won't make a real difference on your hard drive as they will compress practically to zero. However in video memory such compression is not available. Even if texture compression is available it is neither lossless nor is it very strong (50% gain at the best). So in video memory it does matter whether those red areas are stored or not.
Last Sunday I found the time to do some tinkering. TGB's image map system already is pretty nifty and if you hook into the place it is just a minimal code addition that has the task to find out the relevant area of every frame and then store its size and position relative the original size and position. The rendering code can then use this information to simulate the transparent areas of the image by just offsetting the smaller green area in the sprite. It took less than 2 hours to get a prototype working and the result is quite pleasing :)
BEFORE
AFTER
As you can see the sprite sheet created without this technique was sized 1024x256 pixels whereas the second sheet is sized only 512x128. That means it uses only a fourth of the memory the other sheet uses without losing any information.
And that gain can be achieved without any intervention on the side of game developer. Everything happens behind the scenes.
I think this would be quite a nice addition to TGB's image map system.
About the author
#2
It is not really a resource though. I just hacked together a proof of concept. Unfortunatly a modification like this cannot not be made by just adding a new source file to the project. Existing source files have to be modified (t2dImageDataBlock.h, t2dImageDataBlock.cc, t2dAnimatedSprite.cc, t2dStaticSprite.cc, t2dTileMap.cc).
02/13/2007 (6:08 am)
Yes, there is only a small change in the t2dImageMapDatablock code and the rendering code. No need to edit any of your graphics.It is not really a resource though. I just hacked together a proof of concept. Unfortunatly a modification like this cannot not be made by just adding a new source file to the project. Existing source files have to be modified (t2dImageDataBlock.h, t2dImageDataBlock.cc, t2dAnimatedSprite.cc, t2dStaticSprite.cc, t2dTileMap.cc).
#3
You seem to have a small misconception about texture compression on GPUs. They way textures are compressed on GPUs is constant rate compression. For example with DXT1 all RGB8 textures are compressed at a ratio of 8:1 and with DXT5 all RGBA8 textures are compressed at a ratio of 4:1. This is regardless of whether the image is all white or random noise (the noise texture will have much more error when compressed though). The reason this type of compression is done on the GPU is because it's much easier to implement fast hardware for these gradient block based compression schemes.
One tool that I really like using is called the compressionator. This tool lets you compare your original image to what it looks like when compressed on the GPU. Then you can decide if the image should be compressed or not. Any savings from using compressed textures would be additional to the memory savings of your tool. http://ati.amd.com/developer/compressonator.html
02/13/2007 (6:34 am)
This looks like a great tool.You seem to have a small misconception about texture compression on GPUs. They way textures are compressed on GPUs is constant rate compression. For example with DXT1 all RGB8 textures are compressed at a ratio of 8:1 and with DXT5 all RGBA8 textures are compressed at a ratio of 4:1. This is regardless of whether the image is all white or random noise (the noise texture will have much more error when compressed though). The reason this type of compression is done on the GPU is because it's much easier to implement fast hardware for these gradient block based compression schemes.
One tool that I really like using is called the compressionator. This tool lets you compare your original image to what it looks like when compressed on the GPU. Then you can decide if the image should be compressed or not. Any savings from using compressed textures would be additional to the memory savings of your tool. http://ati.amd.com/developer/compressonator.html
#4
02/13/2007 (7:50 am)
Good work Michael, with todays requirements for animation, even 2d games still require a lot of memory. If you're creating a quality sprite sheet, before you know it, you have 10 sprite sheets loaded with a 100 frames each, starts taking quite a bit of space. 
#5
Also note that this not only saves texture memory, but also performance. The transparent areas of those sprites consume fillrate and the less of it, the more GPU performance you gain.
02/13/2007 (8:35 am)
@Michael - This is exactly the system that i've been wanting to see in TGB since before we released the Adventure Kit. I'd love to see your code added to the official TGB codebase as well as some optional GPU texture compression. Great work!Also note that this not only saves texture memory, but also performance. The transparent areas of those sprites consume fillrate and the less of it, the more GPU performance you gain.
#6
There are lots of resources that involve editing files instead of just dropping in whole files... I'd be very interested in seeing this posted as a resource.
02/13/2007 (8:42 am)
Quote:It is not really a resource though. I just hacked together a proof of concept. Unfortunatly a modification like this cannot not be made by just adding a new source file to the project. Existing source files have to be modified (t2dImageDataBlock.h, t2dImageDataBlock.cc, t2dAnimatedSprite.cc, t2dStaticSprite.cc, t2dTileMap.cc).
There are lots of resources that involve editing files instead of just dropping in whole files... I'd be very interested in seeing this posted as a resource.
#7
This is exactly the sort of thing that makes the Garage Games community so great to work with, we have an opportunity to build on each others innovation.
I don't even want to begin to recount the amount of sprite work in the TGB game I am currently working on. This would be very beneficial.
Please do share it.
02/13/2007 (9:22 am)
WOW That would be an extremely useful addition! This is exactly the sort of thing that makes the Garage Games community so great to work with, we have an opportunity to build on each others innovation.
I don't even want to begin to recount the amount of sprite work in the TGB game I am currently working on. This would be very beneficial.
Please do share it.
#8
02/13/2007 (9:42 am)
Yeah, post it as a resource, its definitely something that is rational to use in the engine, unless someone can think of a valid reason not to? 
#9
I'd love to incorporate this into TGB at such a time as it is beyond being a proof of concept, please do post this as a resource!!
-Justin
02/13/2007 (10:04 am)
Michael man... You never cease to amaze me; when is it that you're going to be finished with university any way?!I'd love to incorporate this into TGB at such a time as it is beyond being a proof of concept, please do post this as a resource!!
-Justin
#10
02/13/2007 (10:10 am)
*chants* resource, resource....
#11
@Maurice - Thanks for explaining that. My knowledge of texture compression is very rudimentary.
@Tom - I got your email. I'll come back to your offer for sure.
@Justin - University will take another year at least. That is if I stop at the bachelors degree ;)
02/13/2007 (10:46 am)
Ok, I guess I'll clean this up a bit and then post the result.@Maurice - Thanks for explaining that. My knowledge of texture compression is very rudimentary.
@Tom - I got your email. I'll come back to your offer for sure.
@Justin - University will take another year at least. That is if I stop at the bachelors degree ;)
#12
02/13/2007 (1:47 pm)
This is really cool stuff. Great work.
#13
02/13/2007 (2:41 pm)
This looks fantastic -- we've been doing this by hand on our TGB project to some extent, and hard-coding some of the sprite offset values for various animations, but that's a time-consuming and bitrot-prone way to optimize a game's memory. On top of that, I don't think that our way of doing it gets it down as efficiently as this method looks to. Fantastic stuff! I'm very interesting in seeing something like this added to TGB. 
#14
02/13/2007 (3:14 pm)
Awesome work, and I'll leave out the "I'd love to see this as a resource" comment since so many other have ... but had they not, I'd of said it ... ;)
#15
Note: I definitely want texture compression as well. I guess I want the best of both worlds :)
-Andrew
02/14/2007 (8:56 am)
Great stuff. I'm really looking forward to see how this turns out. The biggest question I have is if it works and plays well with filter padding. Note: I definitely want texture compression as well. I guess I want the best of both worlds :)
-Andrew
#16
The image (96x96 px) above results in the texture page below (48 x 48 px cells):
As you can see the padding occurs around the relevant area of the image - not around the transparent border as it would without the resource. So one could even argue that this improves the filterpad feature.
02/14/2007 (10:14 am)
Filter padding should work fine. Here is an example:The image (96x96 px) above results in the texture page below (48 x 48 px cells):As you can see the padding occurs around the relevant area of the image - not around the transparent border as it would without the resource. So one could even argue that this improves the filterpad feature.
#17
Anyway I think this is awesome work and I'd also love to see this become standard in TGB.
02/15/2007 (8:27 pm)
I did some work with texture compression back in the T2D/EA days. I got some DXT (.DDS) compressed files to load and display properly. But then I ran into issues with padded textures. Then.. I realized I was probably in over my head lol and I moved on to other things. It also only worked with OpenGL mode because I didn't(and don't) know enough about DirectX to make that happen. Anyway I think this is awesome work and I'd also love to see this become standard in TGB.
#18
08/08/2007 (4:36 am)
so, did this make it into TGB 1.5.x? or was it released as a resurce?.... cuz, i really think it would be PRETTY DARN handy... i really need somethin like that. 
#19
07/18/2009 (10:10 am)
Do we have this in TGB yet?
#20
no, it is not integrated in TGB. But I have recently ported it to TGB 1.7.4 for someone. I can not guarrantee that it is bug-free and I don't have much time to support it but I can send you the modified source files if you are interested. Just drop me a mail: michaelwoerister _at_ gmail.com
07/19/2009 (4:51 am)
Hey,no, it is not integrated in TGB. But I have recently ported it to TGB 1.7.4 for someone. I can not guarrantee that it is bug-free and I don't have much time to support it but I can send you the modified source files if you are interested. Just drop me a mail: michaelwoerister _at_ gmail.com
Torque Owner Don Hogan