Dev Diary #1 : A patchwork quilt and a save/load system.
by Gareth Fouche · 01/10/2006 (2:22 am) · 9 comments
Alrighty, so the fundamental structure of my game world is a series of areas, you can travel between areas by walking through certain triggers. Think Neverwinter Nights or Baldurs Gate. If you are in the forest, walking along the path, at one point of it you enter a trigger which loads up the next map. Similarly with leaving the city gates, dungeon entrances etc. So my game world is somewhat like a patchwork quilt, the areas are patches, sewn together by these "paths", which transition the player between them. This was easy to set up, theres an excellent resource on this site about "Mission Portals", which gives you this functionality.
I am now working on a save/load system. When I was last developing (months ago), I decided to leave this feature for later, mainly because I thought it would be difficult, and not really exciting to work on. This time round I am working on it first. If I go ahead and develop other features, when I decide to implement it later I would need to build it around how I've implemented all those other features. Fitting a square peg into a round hole. Far better to build the system now, then when I do the rest I can build them with this save/load framework in mind.
I was at first going to do the file io myself, using fileobjects, doing the parsing etc by hand. But there is a much, much, MUCH better way. I can't claim credit for this one, its something I read on Clint Brewer's site, www.unearthedGames.com, in his dev diary.
Basically, if you look at the missionfiles, torque already has the ability to write and read simgroups. It writes the simgroup, and recursively writes all members of that simgroup. Then, you can exec the file to get all that data back. If you want to add fields to the objects, you can make use of the dynamic fields in TorqueScript. Awesomeness. Not only that, all the parsing is handled for you. Want to save some vectors, strings and matrices? No problem, just refer to these fields by their names, Torquescript will do the rest for you. Double awesomeness.
So , for example you can do something like this (note this is not the actual code, just a general idea of what it'd be like) :
This is very easy to do, and lets you use all the functionality built into simgroups/simobjects. I am really gaining an appreciation for those sim-type-objects, they've got nice search abilities built in, the dynamic field capabilities, the saving/loading, the parsing stuff. Very flexible. Booya.
So I've got a basic implementation of that done, it saves out some playerinfo, mostly dummy variables for now, but also the mission and player transform. Combine that with the patches/mission portals stuff I talked about earlier, it means you can wander from region to region, save the game, and when you load it up again it loads you in the right mission, at the right place, facing the right way. As simple as going %Player.setTransform(PlayerInfo.transform); . Nice.
Which brings me to where I am now. That functionality is nice, but I need this to be able to handle saving all the NPCs, monsters, quests, game state flags etc. The way I see it, there are two types of save info. General saveinfo, that is pertinent to any region you are in (gamestate flags, player info, quest/journal flags etc), and region save info. Region save info is all the save info pertinent to a specific region, NPC data, monster data, container data etc. Ie, stuff you need to know while in a region, but there is no need for it to be floating around in memory when you leave it. Say you have 100 monsters in a region, all with health/mana/position etc that need to be saved, and 20 regions, it could add up to a bit of memory.
So my current idea is to have a "temp_data" folder, which is used while the game is running. This, using the above save/load method, keeps track of game data. When you leave a region it saves a file representing that regions data (SpookyForest.dat for example) to the "temp_data" folder, and when you enter a region it looks for a file corresponding to that region in that "temp_data" directory. If it finds it, it loads up the saved data from it, otherwise it initialises the defaults for the region from the mission file, ie creating NPCs at spawnmarkers etc.
When you quit your game, all the files in "temp_data" are deleted. If you save your game beforehand, it copies all these files to your save directory (might pack it into one file and encrypt it while doing so, but that is something to think about later). It will also save out an additional file, "general.dat", which contains all the general info I talked about, player stats, quest flags etc. When you load your game, it copies all the files in your save dir into the "temp_data" directory, and uses them to setup the game. If you start a new game, it will simply run a script to setup the default "general" info, and then when it loads the initial region, since there are no files for it in the "temp_data", it will load the defaults from the mission file.
So this method allows me to only have data pertinent to the region I am in loaded. I haven't implemented it yet, but it doesnt seem to hard. I am interested to see if anyone thinks that this is unneccessary. Am I just making work for myself? Perhaps I should just have a big structure, with all data for all regions, floating around in memory, and save/load it as needed? Am I just making unnecessary work for myself? Or perhaps someone has a better method? Would be great to hear about it ;).
I am now working on a save/load system. When I was last developing (months ago), I decided to leave this feature for later, mainly because I thought it would be difficult, and not really exciting to work on. This time round I am working on it first. If I go ahead and develop other features, when I decide to implement it later I would need to build it around how I've implemented all those other features. Fitting a square peg into a round hole. Far better to build the system now, then when I do the rest I can build them with this save/load framework in mind.
I was at first going to do the file io myself, using fileobjects, doing the parsing etc by hand. But there is a much, much, MUCH better way. I can't claim credit for this one, its something I read on Clint Brewer's site, www.unearthedGames.com, in his dev diary.
Basically, if you look at the missionfiles, torque already has the ability to write and read simgroups. It writes the simgroup, and recursively writes all members of that simgroup. Then, you can exec the file to get all that data back. If you want to add fields to the objects, you can make use of the dynamic fields in TorqueScript. Awesomeness. Not only that, all the parsing is handled for you. Want to save some vectors, strings and matrices? No problem, just refer to these fields by their names, Torquescript will do the rest for you. Double awesomeness.
So , for example you can do something like this (note this is not the actual code, just a general idea of what it'd be like) :
function save(%player, %savefile)
{
new Simgroup(SaveData);
new Simgroup(PlayerInfo);
SaveData.add(PlayerInfo);
PlayerInfo.name = %player.name;
PlayerInfo.health = %player.health;
PlayerInfo.mana = %player.mana;
new Simgroup(PlayerInventory);
PlayerInfo.add(PlayerInventory);
// dummy vars for now...
PlayerInventory.LeftHandSlot = "longsword";
PlayerInventory.RightHandSlot = "tower shield";
PlayerInventory.HeadSlot = "Helmet of Sheer Awesomeness";
SaveData.save(%savefile);
SaveData.delete(); // since the info is safely on file, no longer need this object floating around.
}
function load(%player, %savefile)
{
exec(%savefile);
//you can now get values from the objects that you created above. Execing the file creates all the objects again. :D
// initialise your game objects from the save info
%player.name = PlayerInfo.name;
.
.
.
//free up the memory once you no longer need it.
Savedata.delete();
}This is very easy to do, and lets you use all the functionality built into simgroups/simobjects. I am really gaining an appreciation for those sim-type-objects, they've got nice search abilities built in, the dynamic field capabilities, the saving/loading, the parsing stuff. Very flexible. Booya.
So I've got a basic implementation of that done, it saves out some playerinfo, mostly dummy variables for now, but also the mission and player transform. Combine that with the patches/mission portals stuff I talked about earlier, it means you can wander from region to region, save the game, and when you load it up again it loads you in the right mission, at the right place, facing the right way. As simple as going %Player.setTransform(PlayerInfo.transform); . Nice.
Which brings me to where I am now. That functionality is nice, but I need this to be able to handle saving all the NPCs, monsters, quests, game state flags etc. The way I see it, there are two types of save info. General saveinfo, that is pertinent to any region you are in (gamestate flags, player info, quest/journal flags etc), and region save info. Region save info is all the save info pertinent to a specific region, NPC data, monster data, container data etc. Ie, stuff you need to know while in a region, but there is no need for it to be floating around in memory when you leave it. Say you have 100 monsters in a region, all with health/mana/position etc that need to be saved, and 20 regions, it could add up to a bit of memory.
So my current idea is to have a "temp_data" folder, which is used while the game is running. This, using the above save/load method, keeps track of game data. When you leave a region it saves a file representing that regions data (SpookyForest.dat for example) to the "temp_data" folder, and when you enter a region it looks for a file corresponding to that region in that "temp_data" directory. If it finds it, it loads up the saved data from it, otherwise it initialises the defaults for the region from the mission file, ie creating NPCs at spawnmarkers etc.
When you quit your game, all the files in "temp_data" are deleted. If you save your game beforehand, it copies all these files to your save directory (might pack it into one file and encrypt it while doing so, but that is something to think about later). It will also save out an additional file, "general.dat", which contains all the general info I talked about, player stats, quest flags etc. When you load your game, it copies all the files in your save dir into the "temp_data" directory, and uses them to setup the game. If you start a new game, it will simply run a script to setup the default "general" info, and then when it loads the initial region, since there are no files for it in the "temp_data", it will load the defaults from the mission file.
So this method allows me to only have data pertinent to the region I am in loaded. I haven't implemented it yet, but it doesnt seem to hard. I am interested to see if anyone thinks that this is unneccessary. Am I just making work for myself? Perhaps I should just have a big structure, with all data for all regions, floating around in memory, and save/load it as needed? Am I just making unnecessary work for myself? Or perhaps someone has a better method? Would be great to hear about it ;).
About the author
Recent Blogs
• SoW Weekly Update 10• SoW Weekly Update 9
• SoW Weekly Update 8
• SoW Weekly Update 7
• SoW Weekly Update 6
#2
And its good to know that someone is working on something similar, and meeting success. :)
01/10/2006 (4:44 am)
Awesome, thanks Nick.And its good to know that someone is working on something similar, and meeting success. :)
#3
01/10/2006 (12:36 pm)
I hope both of you will submit resources on this when done. That would be awesome.
#4
@Rubes: IMO, game specific systems like this shouldn't be resourced. Especially for save/load systems as they're almost always built around your specific game. But the ideas presented here would be extremely trival to add to almost any Torque project. It would probably be better if people coded things like this up themselves, as it would be a good learning experience.
01/10/2006 (2:27 pm)
It's nice to read about how other people code these types of things. Just curious, how many objects are you guys typically saving on a populated map? Everytime you save, do you have to iterate through the mission cleanup/istantGroup and pick out what objects you want to save?@Rubes: IMO, game specific systems like this shouldn't be resourced. Especially for save/load systems as they're almost always built around your specific game. But the ideas presented here would be extremely trival to add to almost any Torque project. It would probably be better if people coded things like this up themselves, as it would be a good learning experience.
#5
It's a manual process but I will only need to do this once when I'm done editing all the missions. If you keep things organized in MissionGroup, it no big deal.
Nick
01/10/2006 (3:42 pm)
It really is a great learning experience and it is game specific, I'll try to get some flowcharts out. We haven't populated our maps with a lot of stuff yet. Personally, I don't iterate through anything. I just make sure that I add the objects that will need to be saved in a separate SimGroup (PersistGroup) located in a separate file. So basically I remove them from MissionGroup and add them to PersistGroup which is loaded from a separate file when a mission starts for the first time. This is done for every mission.It's a manual process but I will only need to do this once when I'm done editing all the missions. If you keep things organized in MissionGroup, it no big deal.
Nick
#6
@Rubes : I will try to post up some of what I do, but it is kinda game specific. It works alot like the code I posted up there, the differences mainly lie in when and where to load files. I would recommend playing around with saving and loading sim groups yourself though, before long you'll have a good grasp of it, its actually very easy :)
Like Nick, I dont really have a lot of objects on my map (haven't got to adding NPCs etc yet), so I can't give you specifics yet Josh. However, I would imagine there would be quite a few eventually, and the less save data I have loaded in memory at one time, the more memory I have for the game objects ;).
I just want to make clear that I am not going to save the actual objects themselves to file, but scriptObjects with some data needed to recreate the object.
Say I have an Orc Warrior. I am not going to save the AIPlayer object (or whatever object it is) to file. Just some pertinent info that might have changed between missions. An orc might have a range of stats like Strength ,Dexterity, Speed, equipment, etc. But I don't need to save those to file. All I need to save is the fact that the creature is an Orc, its position is X, it has Y health, and Z mana. Then, when I am loading, I check monster type, call a loadCreature() script based off that monster type (which loads up the default stats for the creature, Strength etc), and set its position/health/mana to the saved values.
The plan (I haven't gotten quite that far yet) is to create my own Simgroups as the mission starts, and add the objects to the relevant simgroup (just like spawnmarkers are in a group in the editor) when they are created, so NPCs might be in an NPCGroup, Containers in a ContainerGroup etc. Note that these objects aren't what is saved in the missionfile. The mission file will store markers, just like in the demo when you have an AI marker which the script spawns Kork at? When my game loads, if its a new game it will run through these markers and create the objects as normal, otherwise it will build the objects based off save info.
01/10/2006 (11:31 pm)
Worked on this a bit last night, almost got it fully working. I had to add in functions to the engine so I can call functions from script to delete files and copy files. It would have been nice to keep the changes to script, but oh well.@Rubes : I will try to post up some of what I do, but it is kinda game specific. It works alot like the code I posted up there, the differences mainly lie in when and where to load files. I would recommend playing around with saving and loading sim groups yourself though, before long you'll have a good grasp of it, its actually very easy :)
Like Nick, I dont really have a lot of objects on my map (haven't got to adding NPCs etc yet), so I can't give you specifics yet Josh. However, I would imagine there would be quite a few eventually, and the less save data I have loaded in memory at one time, the more memory I have for the game objects ;).
I just want to make clear that I am not going to save the actual objects themselves to file, but scriptObjects with some data needed to recreate the object.
Say I have an Orc Warrior. I am not going to save the AIPlayer object (or whatever object it is) to file. Just some pertinent info that might have changed between missions. An orc might have a range of stats like Strength ,Dexterity, Speed, equipment, etc. But I don't need to save those to file. All I need to save is the fact that the creature is an Orc, its position is X, it has Y health, and Z mana. Then, when I am loading, I check monster type, call a loadCreature() script based off that monster type (which loads up the default stats for the creature, Strength etc), and set its position/health/mana to the saved values.
The plan (I haven't gotten quite that far yet) is to create my own Simgroups as the mission starts, and add the objects to the relevant simgroup (just like spawnmarkers are in a group in the editor) when they are created, so NPCs might be in an NPCGroup, Containers in a ContainerGroup etc. Note that these objects aren't what is saved in the missionfile. The mission file will store markers, just like in the demo when you have an AI marker which the script spawns Kork at? When my game loads, if its a new game it will run through these markers and create the objects as normal, otherwise it will build the objects based off save info.
#7
Out of curiosity, you think that you would be saving a lot of memory by not saving the objects themselves but saving the required info to recreate them instead?
Nick
01/11/2006 (6:55 am)
Gareth, an Orc's default values would be stored in it's datablock so even if you saved the actual object's instance you still wouldn't have to save those values I believe.Out of curiosity, you think that you would be saving a lot of memory by not saving the objects themselves but saving the required info to recreate them instead?
Nick
#8
I'd prefer a system that you just have to say "Create an orc with these stats, equip him with chainmail and a large shield", it does the calculation and you can then query it for his armor rating. Its not a value that needs to be saved, because it can be calculated when you load up the creature, but likewise its not something I want to put in his datablock. I can also envision extending this to a kinda random creature generator, where you create a bunch of orcs with random equipment from a list. Heck, you could randomise their stats a bit too, so you get orcs who are tougher, or quicker, etc.
However, it really depends on your design. I want a system which allows you to give different pieces of equipment to the same type of creatures, and for it to alter their abilities. I am planning to combine this with the mesh swapping resouce to allow lots of variations to one creature type, so I can milk each model for its maximum worth. However, for some games, the armor is just part of the model, its really a superficial thing, the stats are what is set in the data block.
I honestly don't know if I will save a lot of memory, it needs testing and tweaking, for now I'm just going to get something working for a prototype and iterate from there, one of the reasons its great to compare notes like this ;) .In all likelyhood, both methods could work, and its just a matter of preference.
01/12/2006 (5:15 am)
Hmmm, you may have a point, but maybe there will be values that aren't stored in the Orcs datablock, ie things that are maybe calculated, but which are still values you don't really need to store? Say for example armor rating. Perhaps armor rating is a value calculated from the orcs armor, dexterity, shield, skills, etc. You could save the value in the datablock, but then you have to know what that value is for all possible combinations (orc with shield, orc with leather armor, orc with chainmail, orc with the "toughened skin" special ability) etc, and if you tweak the armor value of chainmail you need to go back and change it everywhere.I'd prefer a system that you just have to say "Create an orc with these stats, equip him with chainmail and a large shield", it does the calculation and you can then query it for his armor rating. Its not a value that needs to be saved, because it can be calculated when you load up the creature, but likewise its not something I want to put in his datablock. I can also envision extending this to a kinda random creature generator, where you create a bunch of orcs with random equipment from a list. Heck, you could randomise their stats a bit too, so you get orcs who are tougher, or quicker, etc.
However, it really depends on your design. I want a system which allows you to give different pieces of equipment to the same type of creatures, and for it to alter their abilities. I am planning to combine this with the mesh swapping resouce to allow lots of variations to one creature type, so I can milk each model for its maximum worth. However, for some games, the armor is just part of the model, its really a superficial thing, the stats are what is set in the data block.
I honestly don't know if I will save a lot of memory, it needs testing and tweaking, for now I'm just going to get something working for a prototype and iterate from there, one of the reasons its great to compare notes like this ;) .In all likelyhood, both methods could work, and its just a matter of preference.
#9
Nick
01/12/2006 (5:23 am)
Sounds like a good plan for what you want to do. Looking forward to your to hearing how it comes out to be.Nick

Torque Owner Nick Zafiris
The way I did it, and I actually finished it 4am last night (!) is that instead of using a temp_data folder I use Auto Saving instead.
I'm actually going to use your own paragraph to help you understand what I'm talking about and bold my changes:
When you leave a region it saves a file representing that regions data (SpookyForestAutoSave.dat for example) to the saves folder, and when you enter a region it looks for a file corresponding to that region in that saves folder (SpookyHouseAutoSave.dat). If it finds it, it loads up the saved data from it, otherwise it initializes the defaults for the region from the mission file, ie creating NPCs at spawnmarkers etc.
I also use auto save points (triggers) throughout the mission using the same file name (mission + AutoSave.sav).
There's more to it, like keeping track of the mission status, error handling and stuff but this is the main idea. Works for me. I'll try to get a flowchart on my Save/Load system.
Nick