Best method for storing Data
by Colin Richardson · in Torque Game Builder · 08/03/2009 (1:06 pm) · 7 replies
Hello,
I'm thinking ahead here as this is something i don't need to worry about now.
In our game (basically an RPG style) we will have hundreds of interactable objects. The players will trade for these, pick these up or loot them from a corpse.
So i'm looking at how best to store the data.
It seems to be i have 3 options;
Datablocks
Flat text files
RDMS (SQlite, MySQL)
Now i've done research into all 3 and am not sure what method to go. I am in my day to day life a database developer i know SQL very well so i'm inclined to go SQLite, however i've read a large number of postings on this and it seems that it's quite a bit of work to get it working and i'm a big fan of keeping things all script based for the moment.
Then i thought about Datablocks. I still can't figure these things out too well (i've been reading through The Game Programmers Guide to Torque) and so i don't know if these are the best way.
Finally i thought of having a flat text file that i can read from when an item needs to be displayed, i could easily develop a text file and an index to help with the speed of reading.
So am i missing a choice, or which of these are the best.
I am intending at the moment to store the items the characters have in a text file (encrypted) but if i go RDMS i'd switch to that.
My preference is SQLite but i need some help here, does anyone have it working in TGB and is willing to put up with the pain of my questions on it?
Cheers
Colin
I'm thinking ahead here as this is something i don't need to worry about now.
In our game (basically an RPG style) we will have hundreds of interactable objects. The players will trade for these, pick these up or loot them from a corpse.
So i'm looking at how best to store the data.
It seems to be i have 3 options;
Datablocks
Flat text files
RDMS (SQlite, MySQL)
Now i've done research into all 3 and am not sure what method to go. I am in my day to day life a database developer i know SQL very well so i'm inclined to go SQLite, however i've read a large number of postings on this and it seems that it's quite a bit of work to get it working and i'm a big fan of keeping things all script based for the moment.
Then i thought about Datablocks. I still can't figure these things out too well (i've been reading through The Game Programmers Guide to Torque) and so i don't know if these are the best way.
Finally i thought of having a flat text file that i can read from when an item needs to be displayed, i could easily develop a text file and an index to help with the speed of reading.
So am i missing a choice, or which of these are the best.
I am intending at the moment to store the items the characters have in a text file (encrypted) but if i go RDMS i'd switch to that.
My preference is SQLite but i need some help here, does anyone have it working in TGB and is willing to put up with the pain of my questions on it?
Cheers
Colin
#2
Thanks for the reply.
I'm thinking about SQLite as well at the moment, just writing some more GUI's scripts at the moment.
RBMS are quite good if you know how to use them, but you are right about they're not the easy to use inside TGB. These days i'm writing using C# rather than C++ so it's not the easy for me to make SQLite work in TGB. I'm not too sure how SQLite works, but it can't be that different from MySQL/MS-SQL, and if it's secure then that saves me trying to do that.
However as i said i know SQL very much as each day i work with it (MS-SQL at the moment via VFP/C#) so i can help you out if you do want to make TGB work with it.
Let me know and thanks again,
Cheers
Colin
08/03/2009 (2:02 pm)
Thoughts... too!Thanks for the reply.
I'm thinking about SQLite as well at the moment, just writing some more GUI's scripts at the moment.
RBMS are quite good if you know how to use them, but you are right about they're not the easy to use inside TGB. These days i'm writing using C# rather than C++ so it's not the easy for me to make SQLite work in TGB. I'm not too sure how SQLite works, but it can't be that different from MySQL/MS-SQL, and if it's secure then that saves me trying to do that.
However as i said i know SQL very much as each day i work with it (MS-SQL at the moment via VFP/C#) so i can help you out if you do want to make TGB work with it.
Let me know and thanks again,
Cheers
Colin
#3
08/03/2009 (3:50 pm)
Use the database, the other options will ruin you.
#4
I have to say that's made me smile, it's midnight here so thats a great thing to happen!.
The game is already doing that, i'm finding with TGB it's 2 steps forward, 3 back, 3 more forward, 1 back and so on. I've come so far in just 3 weeks of nights working on this and a lot less doing coding, so if a db is the way to go then i'll go down that road..
Cheers :)
08/03/2009 (3:59 pm)
Quote:Use the database, the other options will ruin you.
I have to say that's made me smile, it's midnight here so thats a great thing to happen!.
The game is already doing that, i'm finding with TGB it's 2 steps forward, 3 back, 3 more forward, 1 back and so on. I've come so far in just 3 weeks of nights working on this and a lot less doing coding, so if a db is the way to go then i'll go down that road..
Cheers :)
#5
If you end up shoe-horning it in place, I'd appreciate some hints on a working way to do it. =:)
Thanks!
Jay Jennings
09/24/2009 (4:13 am)
I was kind of surprised that DB access wasn't already in TGB -- seems like the obvious way to store game data. I'm just getting into TGB and was going to start putting data into a SQLite DB so I could use it in my game, but then realized TGB ain't that far along yet.If you end up shoe-horning it in place, I'd appreciate some hints on a working way to do it. =:)
Thanks!
Jay Jennings
#6
As for some "security by obscurity" - you can read XML into t2dScriptObject and before release dump/save such object to someFile.cs, compile this to someFile.cs.dso and release.
09/24/2009 (6:11 am)
We've decided to fourth option - MS Office 2003 XML format. It's flat XML file, easy to be read by TGB's XML methods. It's easily editable with spreadsheet (like OpenOffice Calc) while being text file - so our level designer works with the OOffice and game. And one more thing - PlayFirst's Playground is using same format so distributors are used to it.As for some "security by obscurity" - you can read XML into t2dScriptObject and before release dump/save such object to someFile.cs, compile this to someFile.cs.dso and release.
#7
If you're relying on a server hosted by you, and clients don't store data, MySQL and Postgres are six times as fast (on average).
09/24/2009 (1:07 pm)
If you need to read everything into memory at once, the smallest possible format is what you need. Maybe that can be compressed XML, maybe a compressed flat text file. If you need to conserve memory and look up random data, a database is the better choice. SQLite is fine, and the choice for single-player/locally served games.If you're relying on a server hosted by you, and clients don't store data, MySQL and Postgres are six times as fast (on average).
Torque Owner Patrick W
Dream Build Repeat
Datablocks aren't persistent (unless stored to a flat file).
Flat files are so easily editable.
RDMS are a bit tougher to implement but allow for secure (if done right, which you most like know about already) storage, re-usability (like adding a high score or online registration, etc) but either require an internet connection or a bigger distro.
I'm on the verge of doing a TGB SQLite integration, maybe we can bounce some ideas...
`Patrick