Image Packing - inefficient?
by Russell Fincher · in Torque Game Builder · 08/08/2006 (12:52 pm) · 40 replies
An image dump shows that the following image map, when packed, is seperated out into two smaller images. No problem there, but the second resulting image is twice as large as it needs to be. Here is the original image map:
And here are the resulting images after packing:
Clearly the image on the right doesn't require a 512px width. It would fit fine within a 256px width. Email me and I'll send you the full-res images for reproducing this.
And the datablock:
[i] new t2dImageMapDatablock(shackcutaway2_map) {
imageName = "./sprites/shackcutaway2";
imageMode = "KEY";
frameCount = "-1";
filterMode = "SMOOTH";
filterPad = "1";
preferPerf = "1";
cellRowOrder = "1";
cellOffsetX = "0";
cellOffsetY = "0";
cellStrideX = "0";
cellStrideY = "0";
cellCountX = "-1";
cellCountY = "-1";
cellWidth = "0";
cellHeight = "0";
preload = "1";
allowUnload = "0";
};
Russell
And here are the resulting images after packing:
Clearly the image on the right doesn't require a 512px width. It would fit fine within a 256px width. Email me and I'll send you the full-res images for reproducing this.
And the datablock:
[i] new t2dImageMapDatablock(shackcutaway2_map) {
imageName = "./sprites/shackcutaway2";
imageMode = "KEY";
frameCount = "-1";
filterMode = "SMOOTH";
filterPad = "1";
preferPerf = "1";
cellRowOrder = "1";
cellOffsetX = "0";
cellOffsetY = "0";
cellStrideX = "0";
cellStrideY = "0";
cellCountX = "-1";
cellCountY = "-1";
cellWidth = "0";
cellHeight = "0";
preload = "1";
allowUnload = "0";
};
Russell
About the author
Art Lead at Sickhead Games, focused on dev tools and prototyping, instructor/advisor at several Dallas colleges and universities, Associate Developer with GarageGames, champion of avant-garde game art at uncommonassembly.com.
#22
Also I have to second Michael on his post above. It would be more than just cool if TGB did this, it is very important to the future of TGB.
At the moment the only way to achieve what Michael illustrates is to have the artist put all the images into the same map. There are several issues with this...
1) It's a ton of work optimizing images in this way. You often don't know what the final packed image will look like. So your taking guesses, guesses which take several minutes to put together and test just to find out that it made the problem worse and not better. It has artists spending time doing the work that TGB could do on it's own more efficiently.
2) When you combine images into the same map your assuming that they will always be used together. So if you optimize for the case on 'level 1' of your game, where sprite 1 and sprite 2 are used together... you are then hurt on 'level 2' where only sprite one is used.
3) Combining sprites to optimize the final packed image is directly counter to keeping your art in reusable libraries (like in our kit). How can i provide a huge library of art assets which are easy to add to your game, but not use 500MB of memory? I cannot prepack images together because we do not know in what combination they will be used. The only way to do this is to pick a 'level load' time how packing works.
08/27/2006 (8:32 am)
@Melv - I will be talking to Justin this week and i'll CC you on my needs/wishlist for TGB 1.1.2.Also I have to second Michael on his post above. It would be more than just cool if TGB did this, it is very important to the future of TGB.
At the moment the only way to achieve what Michael illustrates is to have the artist put all the images into the same map. There are several issues with this...
1) It's a ton of work optimizing images in this way. You often don't know what the final packed image will look like. So your taking guesses, guesses which take several minutes to put together and test just to find out that it made the problem worse and not better. It has artists spending time doing the work that TGB could do on it's own more efficiently.
2) When you combine images into the same map your assuming that they will always be used together. So if you optimize for the case on 'level 1' of your game, where sprite 1 and sprite 2 are used together... you are then hurt on 'level 2' where only sprite one is used.
3) Combining sprites to optimize the final packed image is directly counter to keeping your art in reusable libraries (like in our kit). How can i provide a huge library of art assets which are easy to add to your game, but not use 500MB of memory? I cannot prepack images together because we do not know in what combination they will be used. The only way to do this is to pick a 'level load' time how packing works.
#23
I had some ideas at the time but because I didn't actually have the system in-place, I didn't realise how much of a problem it would be.
Of course, now that the system has been in-place for a while, there's an obvious way to achieve the above. In-fact, I just spoke to Justin about it and he thinks it'll bind well with the whole builder concept as well as from pure-script.
The way we could do this would be to add a couple of script functions "beginPack(...)" and "endPack()". What these would essentially do would start the logical packing process and end the process across a bunch of image-maps. As you can see this would enable TGB to lump together the embedded image-maps. The thing that would have to change would be the functions of "preLoad()" and "allowUnload". These would have to be promoted to the "packing" level and not at the individual image-map level.
This approach would work fine but there might be a better way and that is to introduce the concept of a packing-manager. From this manager you could define as many packing-groups as you like (by name). These groups can have the "preLoad()" and "allowUnload" as well as other parameters set against them. It would then be a simple matter to specify a packing-group for any image-map and if specified, that image-map would get packed against that group.
You would still need to issue an explicit "pack('packgroupname');" but this way would release the constraint of having to define image-maps between "begin/endPack" calls. It could also, potentially, allow you to simply add/remove image-maps from the group and recalculate that group only.
The nice thing about groups is that you could lump together things that are going to be on the screen at the same time such as the "player" and "playerDeathParticles" into logically named areas which equate to texture-sets.
On the fact of it, it sounds cool and it should work well under the builder.
Thoughts?
- Melv.
08/27/2006 (11:09 am)
I hear you Tom/Michael. This was one of those features that Josh and myself spoke about way back when. The problem we had originally was that we didn't have any control over when people defined image-maps and there were other problems.I had some ideas at the time but because I didn't actually have the system in-place, I didn't realise how much of a problem it would be.
Of course, now that the system has been in-place for a while, there's an obvious way to achieve the above. In-fact, I just spoke to Justin about it and he thinks it'll bind well with the whole builder concept as well as from pure-script.
The way we could do this would be to add a couple of script functions "beginPack(...)" and "endPack()". What these would essentially do would start the logical packing process and end the process across a bunch of image-maps. As you can see this would enable TGB to lump together the embedded image-maps. The thing that would have to change would be the functions of "preLoad()" and "allowUnload". These would have to be promoted to the "packing" level and not at the individual image-map level.
This approach would work fine but there might be a better way and that is to introduce the concept of a packing-manager. From this manager you could define as many packing-groups as you like (by name). These groups can have the "preLoad()" and "allowUnload" as well as other parameters set against them. It would then be a simple matter to specify a packing-group for any image-map and if specified, that image-map would get packed against that group.
You would still need to issue an explicit "pack('packgroupname');" but this way would release the constraint of having to define image-maps between "begin/endPack" calls. It could also, potentially, allow you to simply add/remove image-maps from the group and recalculate that group only.
The nice thing about groups is that you could lump together things that are going to be on the screen at the same time such as the "player" and "playerDeathParticles" into logically named areas which equate to texture-sets.
On the fact of it, it sounds cool and it should work well under the builder.
Thoughts?
- Melv.
#24
Cheers,
- Melv.
08/27/2006 (11:10 am)
@Tom: Looking forward to your email. It's a high priority at the moment so I'll address it as soon as you send it.Cheers,
- Melv.
#25
Remember that we're working from image maps in a resource... so more than just what is used may be in the image maps. So what if something in a packing group isn't used in the current level? It seems like it would pack it all together although it is unused... again using more memory than needed. It seems to me like i would want to do this...
1) Load the t2d level file.
2) Get all the imageMaps used by the level file.
3) Put all the level image maps into one or more packing groups.
4) Execute packing on those groups.
5) Let TGB do default packing on the other elements.
Also should we worry about animation frames getting packed into different textures? If not then why not put everything into one huge group?
08/27/2006 (11:30 am)
@Melv - Sounding good, but i have questions.Remember that we're working from image maps in a resource... so more than just what is used may be in the image maps. So what if something in a packing group isn't used in the current level? It seems like it would pack it all together although it is unused... again using more memory than needed. It seems to me like i would want to do this...
1) Load the t2d level file.
2) Get all the imageMaps used by the level file.
3) Put all the level image maps into one or more packing groups.
4) Execute packing on those groups.
5) Let TGB do default packing on the other elements.
Also should we worry about animation frames getting packed into different textures? If not then why not put everything into one huge group?
#26
the packing-group concept sounds good to me. For character-based games you will most likely have one image file per character animation ( running, jumping, ... ) which is the opposite of what the image packaging wants to have ( as Tom said above ).
Why not just make the packing-groups SimGroups (contained in the $managedDatablockSet) and have an extra menu in the image map editor for selecting a group from a dropdown menu or creating a new one.
Btw, from looking at the code it seems that image-maps with "preload = 1" can never unload. The lock-count gets set to 1 in onAdd() and can never reach 0. Is that intentional? Or did I miss something?
08/27/2006 (11:51 am)
Melv,the packing-group concept sounds good to me. For character-based games you will most likely have one image file per character animation ( running, jumping, ... ) which is the opposite of what the image packaging wants to have ( as Tom said above ).
Why not just make the packing-groups SimGroups (contained in the $managedDatablockSet) and have an extra menu in the image map editor for selecting a group from a dropdown menu or creating a new one.
Btw, from looking at the code it seems that image-maps with "preload = 1" can never unload. The lock-count gets set to 1 in onAdd() and can never reach 0. Is that intentional? Or did I miss something?
#27
I think that the idea of packing-groups should be at the "t2d level" level so that you can specify a bunch of image-maps that you require. Of course, the "t2d level" is a construct of the builder but you should certainly be able to create, assign and destroy these groups if you require.
From a pure script point of view it would be helpful if the reference was from the group to the image-map rather than the other way around. This way you could create groups within levels and pick and choose which image-maps you wanted in them and there wouldn't necessarily be a 1:1 relationship between an image-map and a group, essentially an image-map could be in different groups at any time.
It would certainly be helpful if the groups could be linked together so that they could be destroyed in one go. This could be used at level-end to remove the groups quickly.
The potential here, of course, is that two groups are "preload" and they refer to the same image-map which would get loaded twice. This is something that needs user discipline but could be very helpful.
I think that providing the ability to define groups and associate image-maps to those groups is very powerful but can cause duplication of imagery in video-ram if abused but if this is handled centrally then TGB could warning about this.
Regarding the animation frames; you should worry about getting animation frames on the same texture sheet. Getting them to do so is obviously important in reducing texture-swaps and there are a few ways using this packing that you can hint to the system that you don't want (or want warning) if frames are split across pages.
Another feature that I really wanted in a while ago was the ability for the image-maps to cache the best packing solution so that it doesn't need to be done each and everytime. This would save a bunch of time and could even be used to store the packed image-maps if required. This would take more pipeline management from the builders point of view because you would need to specify where the cache was etc. It could even lead to using an encryption system when streaming out the packing images to disk. Obviously there'd need to be a method for designing and publishing image-maps if this was to be used.
Certainly, this system is more complex under the hood and more so from a script point of view but with the help of the builder, it should be easy. Another benefit would be that we wouldn't need the image-map "link" mode which was a hack at best, sucked at all times and continues to suck even today.
- Melv.
08/27/2006 (11:54 am)
@Tom: You bring up some good points.I think that the idea of packing-groups should be at the "t2d level" level so that you can specify a bunch of image-maps that you require. Of course, the "t2d level" is a construct of the builder but you should certainly be able to create, assign and destroy these groups if you require.
From a pure script point of view it would be helpful if the reference was from the group to the image-map rather than the other way around. This way you could create groups within levels and pick and choose which image-maps you wanted in them and there wouldn't necessarily be a 1:1 relationship between an image-map and a group, essentially an image-map could be in different groups at any time.
It would certainly be helpful if the groups could be linked together so that they could be destroyed in one go. This could be used at level-end to remove the groups quickly.
The potential here, of course, is that two groups are "preload" and they refer to the same image-map which would get loaded twice. This is something that needs user discipline but could be very helpful.
I think that providing the ability to define groups and associate image-maps to those groups is very powerful but can cause duplication of imagery in video-ram if abused but if this is handled centrally then TGB could warning about this.
Regarding the animation frames; you should worry about getting animation frames on the same texture sheet. Getting them to do so is obviously important in reducing texture-swaps and there are a few ways using this packing that you can hint to the system that you don't want (or want warning) if frames are split across pages.
Another feature that I really wanted in a while ago was the ability for the image-maps to cache the best packing solution so that it doesn't need to be done each and everytime. This would save a bunch of time and could even be used to store the packed image-maps if required. This would take more pipeline management from the builders point of view because you would need to specify where the cache was etc. It could even lead to using an encryption system when streaming out the packing images to disk. Obviously there'd need to be a method for designing and publishing image-maps if this was to be used.
Certainly, this system is more complex under the hood and more so from a script point of view but with the help of the builder, it should be easy. Another benefit would be that we wouldn't need the image-map "link" mode which was a hack at best, sucked at all times and continues to suck even today.
- Melv.
#28
08/27/2006 (12:01 pm)
I think all of this should happen as transparently and as easy as possible. Wasn't the whole packing system designed so people don't have to worry about image files and image maps?
#29
I haven't really thought about how the classes would hang together yet but I hear what you're saying.
Regarding the "preload=1" stuff; it's intentional as "preload" means always on right from configuration. I'd need to go back and see if there's a problem with allowing unload when using preload. It's been a while since I looked at that code. The name is a little misleading but I copied it from what TGE used for audio etc to mean the same thing.
- Melv.
08/27/2006 (12:03 pm)
Michael,I haven't really thought about how the classes would hang together yet but I hear what you're saying.
Regarding the "preload=1" stuff; it's intentional as "preload" means always on right from configuration. I'd need to go back and see if there's a problem with allowing unload when using preload. It's been a while since I looked at that code. The name is a little misleading but I copied it from what TGE used for audio etc to mean the same thing.
- Melv.
#30
With regards to transparency, all the grunt work would be but the system has no idea of the content of the images so it still needs to be told what groups are logically together. Besides, my explanation makes it all sounds very complex and whilst underneath it is, from the scripts it'd be pretty simple, especially when done from the builder.
I would suspect as well that we could also set-up some nice useful defaults or at least default behaviour from the builders point of view where people simply select which image-maps they want in the level (or the builder just knows what they're using) and it's done automagically.
In the end though, you've got a damn good point in that it would be sweet for TGB to look at what you did and organise the packing appropriately. Josh and myself spoke about something like that quite a while ago but some of the issues were that we'd need to did into the texture-manager which would break TGE compatibility etc.
- Melv.
08/27/2006 (12:07 pm)
Michael,With regards to transparency, all the grunt work would be but the system has no idea of the content of the images so it still needs to be told what groups are logically together. Besides, my explanation makes it all sounds very complex and whilst underneath it is, from the scripts it'd be pretty simple, especially when done from the builder.
I would suspect as well that we could also set-up some nice useful defaults or at least default behaviour from the builders point of view where people simply select which image-maps they want in the level (or the builder just knows what they're using) and it's done automagically.
In the end though, you've got a damn good point in that it would be sweet for TGB to look at what you did and organise the packing appropriately. Josh and myself spoke about something like that quite a while ago but some of the issues were that we'd need to did into the texture-manager which would break TGE compatibility etc.
- Melv.
#31
the preload thing should be stated somewhere explicitly and very big. Just from the name "preload" sounds like "do the work on program start" and not like "...and never ever unload again". Maybe it would suffice to just disable the "allowUnload" checkbox in the editor when selecting "preload"--so people know that no unloading will happen when it is checked.
08/27/2006 (12:08 pm)
Melv,the preload thing should be stated somewhere explicitly and very big. Just from the name "preload" sounds like "do the work on program start" and not like "...and never ever unload again". Maybe it would suffice to just disable the "allowUnload" checkbox in the editor when selecting "preload"--so people know that no unloading will happen when it is checked.
#32
"automagically" is the word. I would not want to have to specify what image maps I use for every level. If that was done behind the scenes without need of configuration on the user side it would be fine.
08/27/2006 (12:12 pm)
Melv,"automagically" is the word. I would not want to have to specify what image maps I use for every level. If that was done behind the scenes without need of configuration on the user side it would be fine.
#33
In the "image-map" doco where all the fields are described one by one, there's half a page of text that describes this and a couple of warnings about it meaning that the textures are always loaded. There just isn't a way to state everything that's important in a "very big" way! ;)
- Melv.
08/27/2006 (12:15 pm)
Michael,In the "image-map" doco where all the fields are described one by one, there's half a page of text that describes this and a couple of warnings about it meaning that the textures are always loaded. There just isn't a way to state everything that's important in a "very big" way! ;)
- Melv.
#34
Regarding your previous post; I agree.
In the end though I'm looking at providing the lower-level stuff that would ultimately be driven by the builder in, hopefully, the most user friendly manner as possible. I'm sure that magical Justin can work some human-factor goodness on this one to ensure you haven't got a whole bunch of hoops to jump through to do something simple.
I'm going to be putting together a document on the features as I see it and then send it along to Justin who I know will circulate it around to everyone. I'll make sure you have a good look-see. If this is to be a major refactoring pass then I want to ensure that guys like yourself and Tom get input.
- Melv.
08/27/2006 (12:19 pm)
Michael,Regarding your previous post; I agree.
In the end though I'm looking at providing the lower-level stuff that would ultimately be driven by the builder in, hopefully, the most user friendly manner as possible. I'm sure that magical Justin can work some human-factor goodness on this one to ensure you haven't got a whole bunch of hoops to jump through to do something simple.
I'm going to be putting together a document on the features as I see it and then send it along to Justin who I know will circulate it around to everyone. I'll make sure you have a good look-see. If this is to be a major refactoring pass then I want to ensure that guys like yourself and Tom get input.
- Melv.
#35
the last time I read this doco was when it came out last November :)
I think it is just a bit misleading to have these 2 options be selected by 2 independent checkboxes. In this case we would be able to state this important not in a "very big" way but just at the right place: Where you select what you want. Maybe have a drop-down menu? So you can't select these 2 at once.
08/27/2006 (12:22 pm)
Melv,the last time I read this doco was when it came out last November :)
I think it is just a bit misleading to have these 2 options be selected by 2 independent checkboxes. In this case we would be able to state this important not in a "very big" way but just at the right place: Where you select what you want. Maybe have a drop-down menu? So you can't select these 2 at once.
#36
Sorry, I didn't realise you meant "big" as in the builder "big". :)
If you send a quick post to Justin/Matt then I'm sure they'll see what they can do. I'm not that close to the builder-work as it stands unfortunately.
- Melv.
08/27/2006 (12:27 pm)
Michael,Sorry, I didn't realise you meant "big" as in the builder "big". :)
If you send a quick post to Justin/Matt then I'm sure they'll see what they can do. I'm not that close to the builder-work as it stands unfortunately.
- Melv.
#37
OK, good idea.
08/27/2006 (12:29 pm)
Quote:
I'm going to be putting together a document on the features as I see it and then send it along to Justin who I know will circulate it around to everyone.
OK, good idea.
#38
Another note... assuming your using texture compression on the video card when available... could the packed image maps be stored in system memory already compressed for the card? It could save a ton of system memory that way.
08/27/2006 (12:32 pm)
@Melv - If the LevelBuilder had tools for specifying how to pack images from that level or even automatically generated packing group information which would then be obeyed when the level is loaded... i think that system would work just fine.Another note... assuming your using texture compression on the video card when available... could the packed image maps be stored in system memory already compressed for the card? It could save a ton of system memory that way.
#39
Yeah, I can do that.
08/27/2006 (12:33 pm)
Quote:
If you send a quick post to Justin/Matt then I'm sure they'll see what they can do.
Yeah, I can do that.
#40
I do need to confirm the work with Justin though.
- Melv.
09/13/2006 (4:06 am)
Got an email recently from Tom asking about this. Just thought I'd post back here and mention that this work hasn't been forgotten about and it's a pretty high priority piece of tech. I've been fixing-up some problems that needed some long-overdue attention and I'd like to get onto this immediately afterwards.I do need to confirm the work with Justin though.
- Melv.
Torque Owner Michael Woerister
These 2 images:
produce these 2 texture pages:
But it would be cool if they would produce one single texture page: