Torque X Builder Memory Limit?
by Matthew Hoesterey · in Torque X 2D · 02/27/2010 (5:49 pm) · 10 replies
So I'm having an issue that is scaring the crap out of me. Torque X Builder is crashing on when loading my game.tx file. (not level just the game) The crashing on startup is getting worse as I add more textures to my project.
What I have found though is that It is sucking about 3 gigs of ram when loading the game.tx file and if I free up ram close visual studio ect.. it opens.
I am thinking that the builder is loading every texture in the entire game when the game.tx file is loaded and can run out of ram.
My game runs 100% ok on Xbox and PC this problem is limited to the builder.
I'm concerned as I have 2 more level files to add the the game and will need to move the entire project out of the builder if I cannot resolve this issue.
Thanks.
What I have found though is that It is sucking about 3 gigs of ram when loading the game.tx file and if I free up ram close visual studio ect.. it opens.
I am thinking that the builder is loading every texture in the entire game when the game.tx file is loaded and can run out of ram.
My game runs 100% ok on Xbox and PC this problem is limited to the builder.
I'm concerned as I have 2 more level files to add the the game and will need to move the entire project out of the builder if I cannot resolve this issue.
Thanks.
#2
02/28/2010 (12:22 am)
Far as I can tell the Builder loads all textures regardless of where they are saved. Am I missing something? 
#3
02/28/2010 (12:29 am)
I have no idea, I was just trying to think of a work around. Maybe start a new project with only the textures you need, and make the levels there, then copy all the resulting files back into your main project?
#4
Anyway, this is how I solve it. It's not ideal, but it at least allows work to continue... In short, you can split the textures across multiple txproj files (there is no need to create a new VS project for this - you just need to a new copy of the txproj with all the materials stripped out). The txproj file is what the builder loads when you tell it to 'open project'. Just copy your existing txproj and cut the stuff you don't need and update any names and stuff in it that need updating.
You can have as many txproj files as you like. All the builder is concerned about is loading a txproj file that you specify and it will load all the assets that the file specifies for use in the builder. And as far as the game engine is concerned it only cares about the txscene files and doesn't know anything about how many txproj files there are. One thing to remember though is to do something like adding a naming prefix to your scenefiles so you know which txproj they should be edited under - you'll want to keep each scene file associated with one txproj for working in the editor.
If needed you can duplicate materials across the txproj files as the game engine will only care about the material references in the txscene files anyway - as far as the game engine is concerned you could have used 1 txproj file or 20; it doesn't matter as it's only the stuff in the txscene file/s that counts at the end of the day.
02/28/2010 (7:03 am)
I have the same issues when using TX. Funnily enough also at about 3GB (I only have 3GB so I assumed it's a physical memory thing, but noted you have 6GB so maybe it is a builder limit).Anyway, this is how I solve it. It's not ideal, but it at least allows work to continue... In short, you can split the textures across multiple txproj files (there is no need to create a new VS project for this - you just need to a new copy of the txproj with all the materials stripped out). The txproj file is what the builder loads when you tell it to 'open project'. Just copy your existing txproj and cut the stuff you don't need and update any names and stuff in it that need updating.
You can have as many txproj files as you like. All the builder is concerned about is loading a txproj file that you specify and it will load all the assets that the file specifies for use in the builder. And as far as the game engine is concerned it only cares about the txscene files and doesn't know anything about how many txproj files there are. One thing to remember though is to do something like adding a naming prefix to your scenefiles so you know which txproj they should be edited under - you'll want to keep each scene file associated with one txproj for working in the editor.
If needed you can duplicate materials across the txproj files as the game engine will only care about the material references in the txscene files anyway - as far as the game engine is concerned you could have used 1 txproj file or 20; it doesn't matter as it's only the stuff in the txscene file/s that counts at the end of the day.
#5
03/01/2010 (7:19 am)
Thanks Duncan Again you save the day :). 
#6
03/01/2010 (8:27 pm)
Wow Duncan are you saying the txproj file is for the builder alone? The final deal doesn't use it at all? 
#7
03/02/2010 (3:10 am)
It doesn't appear in the compiled folder, so it's only used by the builder.
#8
03/02/2010 (9:20 am)
Yep, txproj is just the builder's record of what assets and stuff to load into the builder, and the game itself just uses the txscene files the builder produces. The txproj file isn't even used by VS at compile-time. Which is why you can have multiple txproj files - all the information the game needs is put into the txscene files (the textures to load, animation frames, etc).
#9
It sounds like the builders do indeed load everything up into memory from what you folks are saying. This means, assuming the builders are 32bit apps, there is a hard limit to the size of your game resources (while creating them). The games themselves only load what they need when they need it I imagine.
That's just my thought, though.
If I'm correct then, for really large projects, you're going to either need a 64bit builder or better optimization in either the builder or the resources you are putting into the game.
03/02/2010 (9:47 pm)
If the builders are 32 bit apps then you are approaching their memory limits regardless of whether you are running a 32bit, 32bit with PAE, or 64bit OS. The max any 32bit app should be able to address is around 4gb total. Even with PAE the individual apps are still limited to 4gb pages.It sounds like the builders do indeed load everything up into memory from what you folks are saying. This means, assuming the builders are 32bit apps, there is a hard limit to the size of your game resources (while creating them). The games themselves only load what they need when they need it I imagine.
That's just my thought, though.
If I'm correct then, for really large projects, you're going to either need a 64bit builder or better optimization in either the builder or the resources you are putting into the game.
#10
As for the scenes in the game only loading what they need... unfortunately that's not the case. The builder saves references to ALL materials that it has loaded. And the scene file will dutifully load them in the game. This is why Matt made his material culling tool to strip out unneeded materials from the scene files.
03/03/2010 (10:33 am)
Doh, why didn't I think of that! :) Yep the builder is a 32bit app as far as I know. An additional note for people with 32bit XP is that the Graphics Card memory counts as part of the 4GB that XP can address - so even if 4GB of main ram is installed it won't use it all.As for the scenes in the game only loading what they need... unfortunately that's not the case. The builder saves references to ALL materials that it has loaded. And the scene file will dutifully load them in the game. This is why Matt made his material culling tool to strip out unneeded materials from the scene files.
Torque Owner Trent