Basic mo/mmo client/server data base with the Torque 3d Pro

#1 I think what you're really aiming for is something more like Neverwinter Nights - a multiplayer-capable RPG. Not as complex as a full-fledged MMO, but still not for the faint of heart.

The main thing to understand here is that even "fps shooter type" tutorials get you working in script (and sometimes engine source), so there is very little that is not useful to at least read through. Having said that....

Perhaps start with the RTS Prototype (documentation->Scripting->Advanced folder) and then proceed to the Adventure Prototype (in the same folder), then you should perhaps think about looking into the server connection system. There are a few forum threads from back a bit discussing this, most of that information should still work. I would also suggest completing the FPS Tutorial while you're at it - some good basic information there and plenty of scripting/editor practice.

The way I think you'd want to go about it is to use the built-in server selection system. You also want to have a way to assign a client ID so that once you connect to the server you can figure out which saved character data belongs to whom (and also add a GUI and back end for sorting that all out).

Once you get a basic setup working so that it can sort out who's who and get a simple weapon/ammo loadout working you can expand that to just handle more stuff using flat file storage, or you can delve into adding a database to the engine. Generally people have used MySQL or PostGRE SQL, but there are a few options available.

So, by the time you get through the homework assigned above hopefully others will have chimed in here with links to other useful threads, blogs and resources to help you on your way.

#2 @Richard Ranft those links gives an

Quote:Application Error

Edit: Btw Zeph: Are you willing to throw some money at it? Because Kevin Mitchell's RPG Engine might be a good place for you to start:
His latest blog
Otherwise, if you want to develop everything yourself for the sake of learning, I would like to know more about how many players you aim at? If it isn't too many, the built-in server might be enough for your game (maybe shred out some of the FPS-oriented stuff)

Generally look at Kevin Mitchells resources he have made some great RPG-oriented ones (including character creation and saving it seems)
Kevin Mitchells resources

Here is a save and loading system that shouldn't require any engine changes:
Basic Save and Load Variables to a New File via FileObject, by Steve Aacaster

Another one which seems more extensive by Steve Aacaster

#3 If you are interested in learning your way around theory of MMORPG creation, the Prairie Games Torque MMO Kit might be worth looking into.

www.MMOWorkshop.com

There is a T3D version, but the condition of it is rather rough. The versions based on older Torque engines are much better. My suggestion would be the one based on TGE 1.5.2 as being the most feature complete.

Both client and servers are Python applications that use a modified Torque as a rendering / simulation library. World design is handled from inside Torque's world editor. RPG design is handled in Python through the genesis system which creates SQLite databases used by server and client during gameplay.

To see a complete game built on this framework, take a look at A Mystical Land.

www.mystical-land.com/us/

Be aware that there is little, if any, continued development or support of the kit's software other than a few remaining devs (mostly hobbyist level) that you'll run into on the forum. So, for the most part you'll be on your own in getting it setup and working (not an easy task in itself).

The biggest advantage of the kit is that the base framework for most MMORPG functions already exist. If you are familiar with the original EverQuest much of the design will be quickly recognizable. Using it you should be able to self-educate on the game mechanics involved.

Beware of a steep learning curve and expect plenty of headaches especially if you want to build features not already provided. A programmer comfortable with Python and C++ is essential for anything other than basic modifications. If development work is an option, then possibly bringing the T3D version of the kit from 1.1 beta 1 to 1.2 MIT and fixing the broken functionality could be a practical direction.

#4 "Fixed" the link issue - once upon a time you could navigate directly to the docs pages, but I suppose we've got some weird redirect issues going on at the moment.

#5 Ok so Ive been away for a few days downloading, and messing with various tutorials like mmo workshop which doesnt seem to work with the 3d version.. Idk why I followed the install pages...

I went through the engine code looked at the way the engine connects to servers, and the server listing, and how a player would host their own game.. I dont want a server listing where a player can host their own server that destroys at map session end that restarts on next server map instance.. Im aiming more of a one server instance that is persistent, that saves data to a data base, which I know torque can handle Ive looked at the code, just need to do some future re-editing how the client connects to the server...

#6 Richard Ranft.. What Im more or less aiming at some day is a full MMO, how many players will depend on how successful the MMO is...

My aim is to create a open world, persistent, sandbox game that give the players all tools they need to craft, and create whatever their little sandbox hearts desire..Weather it be a hermits hamlet to a full scale city, allowing players to choose where they wish to congregate, while living in the environment I create, with its own dictionary real world knowledge base... I dont want to release too much of my ideas since non of them are protected yet. Think in terms of Tales in the desert,and Roma Victor with a sandbox style world building, and editing. Not WOW or EVE, However I would like to incorporate the Eve knowledge type system for learning, to advance skills..

Im kinda taking the Runescape approach starting small in level design since I am poor. Andrew the creator of Runsescape was pretty much in the same situation. However his strong points were coding, and lacked the art skills.. In my situation Im an artist, and certified with noob code skills.. I remember when Andrew created Runescape, he started with one city, one server small data base with a few items, and horrible graphics for free with a hand selection of alpha, and beta sign ups.. I was one of them.. He slowly expanded on over the next year to include for a total of 3 cities, and started to charge $3 a month.. Some time after this point a couple years later after saving up the capital from the $3 per month, he expanded, and hired staff for better graphics slowly... Now he charges along the lines of close to $7 a month. So I do know that creating an mmo is possible even starting out alone.. Andrew is one testimony to that...

Im pretty much trying to recreate his approach, while I learn how to design, and make my game available..However, in order to do this I need to learn how to actually make a server, log in system, with a data base.. Something that I can use and learn from on a small scale on my own home machine... Im just confused as to how it all connects, and what to use, and how to use that to implement that.. I know if I can accomplish, a basic server log in, and data base save system, I can accomplish the rest which is easy since Im an artist in heart...Im quite confident, and self taught in the area of making models using the world design editor.. Ive used several engines testing,and learning from which I prob should not have done since it kinda wasted my time thought out the last 13 years jumping around from engine to engine. I always find myself comming back to Torque because Torque is pretty much exactly what I want, and offers the tools I would like to implement inside the game... Ive been teaching myself programing slowly over the years as a hobby, fiddling, and messing with code educationally.. Im not no code guru, however I do understand the very basic principles..

Im just more or less at that stage of learning how to make that server, login database.. Where I can develop a system to get users to connect to the game world, and save that world.. For now, I just want it to be me, and maybe someday include a few friends to test features as I develop the world.. But Spending time at this point in developing a world without that login database server is pretty much a waste of time.. Once I have solid structure, security, and some content created, Ill release into an alpha, but thats far off into the future..

This last week I spent some time on various tutorials, downloading different software, testing out two servers like the Apache or Microsoft ISS server, which both use Mysql db, and php.. I played with some tutorial test scripts on login db on a web page on my local host with registration, email confirmation, user data base setup... THe Microsoft server was very difficult to use, too many application installs, but seemed more reliable then the apache, but I just want to learn the basics for now, so Im currently sticking to simple with Apache, Mysql, and Php, PHP MYADMIN with SSL and SHA1 for security.. Unless someone can point me into a different direction..

Im understanding more of how a data base structure is, and user log in is all put together now on a website, which Im sure the ideal is pretty much the same for a game login using the same database principles.. I just need to figure a way of implementing this structure with Torque engine, and its engine code..

Im open to code snippets as examples, or tutorials that can do this.. Otherwise google will be my best friend till then.. :)

#7 I just wanted to take the time to update you guys.

I finally figured out how to database, and get a basic server going. I also now understand fully how it works, and what needs to be done. Just a matter of coding one to work within Torque script, and getting it to pass or retrieve the info between client, and server.

#8 @Zeph,
I would be interested in seeing you blog about your experience with creating an MMO framework or using an existing one. I could see a blog like this having a lot of value to people to see the design behind the tech, the pitfalls, the eurekas, etc. I have an interest in this subject and I certainly would read the blogs. It would also give you some higher profile in your progress and some of the MMO experts around here would be able to chime in on from time to time.

A while back I took a gander at the MMOWorkshop design pages and learned a lot about how the server systems are designed. It was quite insightful to see how their particular system is setup.

For a project I have going now I am thinking of using a loose MMO concept where missions reside on machines hosted by individuals, but report back status to a primary server. Due to the design and focus of this particular project data integrity (people being able to cheat) is not as critical as it would be in a full MMO. But any discussion as to the design behind MMO I am all ears. I can see this project potentially morphing into a full MMO design. Time will tell.

#9 @Frank:
The challenge behind the hosting on the players is the fact that Torque gives the end user access to the console by default (and even if you remove it in script, it wouldn't take much to get it back).

With that in mind, it introduces a whole new level of problems with using TorqueScript to handle the more complex tasks that would otherwise be extremely tedious to do in C++.

I have the same issue with my current "free" zombie survival fps. I have a basic weapon progression system inside the game, but it has to be exposed to script at some point, which from a security perspective, is only going to be problematic once the console can be enabled.

@Zeph:
The challenge with designing an MMO with T3D is the fact that TorqueScript itself is substandard to C++ in terms of processing speed. So you won't want to handle your calculations with it (having many players connected at once to receive said calculations will only introduce problems), so a majority of your work is going to have to be internal (C++) to prevent a massive performance hit on the server.

Secondly comes around the fact that you will be dealing with a large amount of players at one time, and from what I recall, the standard T3D build can only support 64 players at once on a server without a major network hit, so you will need to optimize the methods of pack/unpack data to accommodate for larger player counts.

Lastly is security and account integrity. This will take you quite some time to code up, and debug (as I have felt the pain of for 1 and a half years with my account system) to ensure it works. You will need to pick a system that safely stores data on the server, but doesn't take a massive amount of processing such to create issues when multiple people are using the script at once. You might want to take a look at my account system resource if you get the chance. I use a certificate based system that keeps the necessary login information on the client side such that no transmission with the server are needed unless you are either registering for an account or recovering your certificate from the server (need this or else this system is just a complete failure).

If you can get past all of these hurdles however, I see no reason not to make a successful MMO using T3D. Keep us updated, I'd really love to see your progress on this.

*Edit*
I really need to read more in depth then scanning over posts, lol.. Zeph, I'd really take a look at my XXZ568 resource if I were you. This has all of the necessary scripts to create accounts using RSA-768 key-pairs, AES-256 Password Encryption, and Data Hashing under SHA1/Whirlpool. It's a very powerful system I based off of the method done by the TribesNext Community project that re-enabled multiplayer for Tribes 2.

I will be updating the resource for the T3D implementation in a few days (after I get through final exams :x) to address a problem with the TCPObject communications (you can see my thread regarding that), but other than that it should point you in the correct direction for getting your own system to communicate with your server.

#10 @Robert,
I won't be using Torque Script to talk to the data servers. I will be using Python. It has a large selection of industrially/commercially proven networking libraries available. This is one of the reason the MMOWorkshop folks went with Python as well.

It is trivial to do callbacks from TS to Python and vice versa. So you could potentially add Python to an existing project and have Python handle the painful network communication. It also has the ability to process complex data structures like XML, JSON, encryption, etc.

I don't foresee the data servers being based upon T3D either. The networking between clients will be T3D, but the data servers don't have to be.

#11 this has been the goal of what WLE is doin, setting up the whole framework for the MMO. That was part of the original reason for us coming up with DNT and the reason why we are working on all of the other parts and piece we are now. TorqueScript was just too slow. However, using DNT and the .NET framework was a huge difference.

As far as the DB side goes, we are having someone take a look at that now. not just from a game play perspective but also a data entry side. Its a rather interesting project. Every time that we think we have most of the fields worked out (even on an extremely basic level) another 1 or 2 pop up.

#12 Paul,
I can't wait to see what you guys come up with. Are you guys going to be able to run on Linux servers for the MMO servers?

#13 @Frank

Everything that we are working on now we are making sure is Linux compatible just for that reason. we REALLY need to get a dedicated Linux guy inhouse to speed up that process though. Its something that we are looking at doing but the right candidate hasnt come across my inbox yet. Until then, we are still going to get the Windows up and running and make Linux available when we can.

#14 Robert. I understand what you are saying. In fact I downloaded your resources few days before your post, and I couldnt get them to work with the Torque 3D.. I thought they were used with an older engine. Your resources did help in my research into making a server, and I did like the idea about the SHA1 data hash. In fact it lead me to learning about PHP, and Mysql, login templates both on a website, and for the game login, where the gui is involved.

Right now, Im not trying to go big, probably not even more then 2 people on my server. Ive never made or ran a server before other then the old UOX Ultima Online precomplied servers out the box.

Im basically setting up a demo world, that I can log into, and test to learn from how servers are made, ran, administrated. I want to do it this way so I can learn from scratch, and know what I need to do when I do go big. But, when I do go big, I will still start relatively small as runescape did. I dont see myself releasing the game fast. I will milk the cow so to speak. As the money comes in, I will grow, and add more server space.

Down the road Im not too worried bout a larger audience, and a larger server. My game concept is a sandbox environment with many, many real life maps, which will probably be on their own servers. So, Im talking bout a lot of small server systems in a network cluster. Players might not even bump into one another for awhile unless they are in popular areas, or choose to congregate build in an area.

I think between servers, per map areas, and the size of the world server loads should be fine under the 64 limit. I did think maybe, for the areas where players want to be in over 64 limit say a store trade area. I can assign multiple servers with the same exact copy of that map where the server can check, and assign a player going into that area to the next best same map if the 64 limit has been reached. I was even tossing the idea around on free open trade, that players make, and define the economy so really there is no reason to congregate in one area or get into npc shops. I really enjoyed the sandbox element to Tales of the Nile, except there was no real hunting, combat, aspect to it, and the building tools fell short. The player knowledge in depth was fantastic, but I think it can be expanded on into a wider skill set for players more Eve style where there is not one jack of all trades player being self sufficient.

To give you an idea on world size, Google Tales of the Nile, or Roma Victor. In those games you barely see many people in one area. The most players you see in one map is probably 1-30 in one real life day because the world is so big the player base is spread out over the server tiles.. So Im not doing no WOW Online, but there are over 1000 map tiles to exlore, and build or congregate on..

I am however looking forward to your update. Oh Robert, I wanted to add, I was thinking about a zombie/survival shooter game (also) using what Torque already offers in client server architecture to raise funds for my real project. I dont see anyone doing zombie vs players allowing a player to play either one. I thought It might be interesting to implement this. Ive been following other zombie games, and I see a lot of players asking to play the zombie, but no one is making this kind of content. All I really need is to do is the ART assets, and some minor scripting changes. I'm just deep in thought cuz I kinda had an idea in mind of even adding a database something how Robert was mentioning, but just for skill sets,and leader boards with maps always resetting. Maybe even just forgetting the database, and just allowing roll in skills at map selection based on the type of character they pick, ie lawyer, construction worker, doctor, boxer, military ect...

#15 @Frank,
I have no problem making a blog, on this subject when I actually accomplish what it is I am attempting. I would also like to know 100% of what Im doing, before I release a half done blog on info... I might even make a tutorial. Im just picky when it comes to info being correct and in order, instead of being one big blob mess all over the place, and people will be like huh?

Im sure you know what I mean because Ive read thousands of pages just on this subject alone that left me saying huh???!!!#$%%$#

Because... the information just didnt work, or they didnt explain it right, or its just way too out of date... I hate half ass tutorials/info done incorrectly.

For awhile I was really confused between database server, and game server. They are actually two different things. Torque engine already has the function to create a game server. What Torque lacks is a database server, and functions made to communicate between the two on saving players data, inventory, items, skills, log on log off locations, and whatever else you can think of that goes into an MMO. Torques server architecture is designed for MO style shooter games, where the server resets, and a player logs off refreshing the game for a new map. Just need to edit the Torque scripts, and engine code as to your needs.

For now what I can say is. My approach is based on a free approach until Im ready to actually release the game world live. In order to do this. Im using Torque GUI, MYSQL, PHP, PHPMYADMIN, and SHA1. Ive found out its pretty much broken down into two sections.

1) Create one database with MYSQL, PHPMYADMIN for a website. Use PHP to script login pages for your web site. Save the website customer info to the database.

2) Create one database with MYSQL, PHPMYADMIN for the game. Torque already has code to make the server to play on. You will save the player info from the game world to this database. Use the Torque GUI to create your login pages into the game.

Note: For now Im using one database for web, and game since its free, and on my same personal machine (localhost), and just for me. I still need to connect the two together in script. Im just kinda back and forth on projects of learning terrain tiling, character creation, making the real life world maps from DEM's. Including spending the last week reinstalling My O/S, and restructuring the the hard drive, for being able to use my computer for both personal, and game design.

If I ever go live, I will use a paid web host, and have the paid MYSQL service, and use a merchant account for any payments. I will probably create a central hub server to communicate between both the web host database, and the game server database for security reasons.

#16 @Zeph,
Pay attention to Robert Fritzen's posts on networking to the database/servers using TS. He is pretty much an expert from seeing what he has done TS networking.

If you want you can look at doing that communication using Python using the ScriptT3D resource. That is the path I am taking. I don't have code demonstrating that yet. Most of the Python networking code I have worked on was for customers. The default Python libraries hold up really well in this regard both as a server and/or client. I am also not working on an MMO at the moment.

Yeah, I hear yah on the tutorials for different things. I am working on an FFT implementation (using the KISS FFT lib) and it took a good few hours to figure things out from looking at the web pages. Finally clicked this morning when I got it working. Mainly the barrier was the jargon with the math terms. I am guessing for MMOs they have their own lingo and probably have borrowed from game jargon and IT jargon.

#17 @Frank. I am paying attention. It was Roberts stuff that actually got me to the point I am at lol... I already got the database working/server working due to his help resources.

You say take a look at using Python, wouldnt that be pretty much the same as using Torque script for the server leaving a person in the same situation as substandard to C++ in terms of processing speed when the server handles clients, and taking its hit toll?

I know Ill have to edit the source eventually, and Im ok with that. I will do this eventually, when I fully understand what Im doing with server code, and when its more of a natural state of mind. Right now Im learning with what Torque has, and offers in making a developing test server with the .CS script files.

I think Im in the process of making two executable, one for the server, and one for the client. I know this will take extra work, but when done will make things a lot easier down the road by simplifying. This means I have to restructure the folders, and files. When the restructure is complete I can eventually rewrite the server script files into C++ in the source, and compile. Well when I feel knowledgeable or comfortable enough to do so.

I actually started working on the folder, and file restructure today. This is going to be a hair pulling mission for me lol.