Client / Server Separation and optimization

#1 ...
make dedicated

#2 First format your Hard Drive to rid it of the horrible bloatware virus known as windows, (sorry you contracted that on your server), and install Linux, prefferably Gentoo or some other source based distro that will allow you to custom compile everything you need and ONLY what you need w/o going to dependency hell. Make sure all of your CFLAGS are set properly, so you get exactly the right build for your hardware.
After that simply go to where you have loaded TGE and

make -f mk/configure.mk OS=LINUX COMPILER=GCC3 BUILD=RELEASE
make --dedicated

This will make the leanest meanest game server you could possibly imagine.

#3 Not sure what to say guys. I was hoping to remove modules like sound, GUI etc.. I'm aware of the dedicated build but maybe I dont understand what it does Stephen. Could you elaborate for me?

As for Windows, too each his own. Not many people have as much MS Windows backend experience as I do and for server farm administration purposes at this time I can do far more with Windows (as I stated originally) than I can with RedHat (Which I am running a trial server on for possible production migration). I am not for a moment stating anywhere that Windows is the more stable or efficient O/S but for development I have 15 years of professional IT management skills that give me the edge I need to make this work without hiring unix support people. If you are a unix master, great more power to ya. I definitely, am not. I know that and I am developing appropriately.

As much as I can read up on the server options etc I was really look for some human interaction with someone who has stripped down a dedicated server in Torque before. Not really looking for suggestions or speculations ;) Got enough of those thanks.

Seriously though if anyone has any development time optimizing the dedicated server code I'd like to talk. I think Artifex has gone through this. Perhaps I'll drop him a line tonight.

Mark

#4 The dedicated build rule removes all of the modules that don't apply to a dedicated server--like sound, rendering, graphics, etc. For reference, my DEBUG dedicated server is about 20 meg, and my DEBUG dedicated client is 31 meg. And the server build has additional files that the client module does not.

In other words, what you want to do is what "make dedicated" is for.

#5 Stephen thats great info. thanks man. Obviously my complete lack of coding comprehension left me in the dark as to what the dedicated build did. You cleared it up 100% for me. Thanks a mill for the explanation. Hopefully my ignorance will at least help other folks who might have gone down the same path as me in trying to manually extract the modules haha.

Cheers man.

Mark

#6 Stephen thats great info. thanks man. Obviously my complete lack of coding comprehension left me in the dark as to what the dedicated build did. You cleared it up 100% for me. Thanks a mill for the explanation. Hopefully my ignorance will at least help other folks who might have gone down the same path as me in trying to manually extract the modules haha.

Cheers man.

Mark

#7 What would you change, and where to compile a stripped down dedicated server using VC++ 7?

#8 Does anyone know how to build a dedicated server using VS.NET (2k3)

or should i open up hte make file, find hte rule for dedicated and start a new project..

#9 I like that bit of knowledge as well. Any codehead know what to change in Torque Demo.vcproj?

#10 Wish I could help guys, but I don't use either of those platforms. I think Pauliver is on the right track, but that's just from general knowledge, not from any specific knowledge of the IDE's you guys are using.

#11 Thanks again Stephen. Just wanted to say that after4 months off I spent a few days puttering with my old code and now have 2 game servers and 2 auth servers running for over 2 days straight now.

I log in from my dev machine a few times a day just to kill Korkie a few times =)

I have some bandwidth and process monitoring software running on the servers and will analyze the results after a few more days but so far my persistant world is.. well.. persistant ;)

I have some more error checking to work on on the auth servers and then I'll start adding NPCs into the world.. Mmm.. Fresh Meat.....

Mark

#12 Sounds awesome! It's good to see someone stay the road and do the background research needed to accomplish something that sounds difficult, but if you take it a piece at a time, can work really well with TGE!

#13 The makefiles are all plaintext. Worst case, you can just read through them, see what files are being compiled, and set up a project in your IDE to do the same thing.

Oh yeah - dedicated mode is also driven from the command line when you launch the game binary. On windows and so forth you just run mygame.exe -dedicated -mission foo and it will automagically be in dedicated mode. Thanks to VM, unused code will get paged out to swap.

#14 Thanks ben.. all of that is nothing new, however i was hoping that there would be an easy way to compile a dedicated only .exe [one that didn't requre mindless grunt work of makefile to vcproject] for the simple reason that i run my server over remote desktop, and running torque causes a memory leak over it [when in gui mode] so i was hoping there would be a dedicated server easily compileable under windows to prevent me from ever accidentally double clicking it.

The information on that the VM will page out unused code to swap memory was something i didn't know, very usefull thanks


On a seperate note why resources did you guys use to make your worlds persistant. i found MANY threads out there that talk about it, but no where did they say what variables/code was changed for it.

#15 @Paul: I've made a couple changes to the codebase for the auth server but nothing major. Just tweaks for the database. Other than that, currently all my servers are Torque scripted. Thats the best way to develop in my mind. Get all my proof of concepts out and testing. Then once they work the way I want them I harass some poor programmer to make them into C++.

The only thing really persistant about my world right now is the player DB. Naturally it doesnt really matter because the players aren't doing anything other than rack up kill points on Kork =) but hey, thats still something isn't it.

My plan is to get my auth/account server code solid. Check it for basic loopholes then start dumping some more difficult persistance in like roaming camps of AI enemies, much more defined player statistics and interactive NPCs.

You know my email, drop me a line if you have any questions =)

Mark

#16 @Paul: GUI leak? But how can there be a GUI leak when you're running Torque in dedicated mode?

#17 @Ben: sorry for bieng unclear on that. the Gui leak occures
when i accidentally just run torque [in regular mode, forgetting to run it in dedicatd server mode] over remote desktop, on an Itanium based server.

#18 @Paul: I would bet dollars to doughnuts the mem leak is in RDC heh. Depends on the O/S though. I've had RDC open overnight many times. Wasn't running anything nuts with it though. However I got to tell you running Torque through RDC.. ouch. Yeah I can see memory issues with RDC doing that.


Anyways all I am going to post an update for any poor unfortunates as stubborn and pigheaded as myself who attempt to strip down the starter.fps demo instead of starting from scratch.

This is not an easy task so here are some tips:

A) Keep a "Master" directory of the unblemished SDK structure with all files and NEVER touch this "master" directory except to "copy" files. You must have a 100% unblemished point of reference for when you get stuck.

B) Keep in mind that Torque script is *very* structured and you will find main.cs, game.cs and several other files spread throughout the directories that are called under certain circumstance. Learn when they are called.

C) TEST AFTER EVERY SINGLE MODIFICATION ;) All your hard work can be destroyed if you forget to test for 10 mins and find you have blown up the client.

D) After every successful "major" modification I moved my "working" copy of the client to a "production/client" folder. This is, like the "master" copy, untouchable. Saves losing alot of work with an accidental delete.

E) Knowing that most of the functions and GUIs are called at runtime, find the source of the calls for the directory(s) you wish to move and see what the best way is to move the source of the call. (EG: I found that by renaming common/main.cs to commonmain.cs and moving it to the root of starter.fps I was able to migrate a TON of the common folder info to /client inder FPS.)


Conclusion:
This takes lots of work guys but short of starting from absolute scratch with nothing more than the SDK source it's definitely the way to go in my books. Torque script is wicked powerful and although I only really started puttering around with it a few months back, I assure you that if you stick to it and don't let any initial frustration get you down you will prevail ;) It took me about 4 hours tonight to clean up my authentication code and add some decent level of errorchecking but it was well worth it. Sure I could've done it faster if I knew what I was doing ;) but thats the beauty of Torque. It let's amateurs like myself inflate their egos by producing some stunning results in a pretty miraculous timeframe.

Cheers,
Mark

PS: Quick edit: I just wanted to say that in about 2 hours today I managed to strip down my client to reside entirely in the starter.fps folder. It contains no servercode, no editor code and connects to my game servers no problem. Yes it was frustrating, no I wasn't impressed with my choice of words on occasion, but the fact is I strongly recomend, actually I insist that if you are even remotely serious about using Torque in a C/S environment you do this exercise. I learned so much about the internals workings of the script tonight it's not even funny. Too bad I'll forget it all tomorrow ;)

#19 Set a breakpoint in GameConnection::writePacket a step though it. you'll be amazed at the amount of belt tightening you can do. also rewrite BitStream::writeCompressedPoint its inefficient and buggy. if you have time go thought as many ::packUpdate and ::unpackUpdate as you can. you'll find that people tend to send stuff for completeness not realizing that it doesn't have to be sent, sometimes data is sent to the client and is never used.

#20 To follow up on Cameron's point, a lot of resources and code snippets that are released are not necessarily optimized for network traffic--many of us may tend to not test them all the way down to the minimum number of bits required, so it might be good to work through pack and unpack for any resources you choose to implement with a close look at optimization.

Of course, some are better than others!