TGE Simplification - Part 1: Merging /Common

I see a lot of people asking how to streamline and simplify the TGE script codebase. This first tutorial shows you how to merge the /common directory, and future tutorials will build on this to merge /starter.fps, and then go on to show you how to eliminate packaging and module functionality from the TGE scripts to further simplify the TGE script codebase.


Starting Point:
---------------
These tutorials will all build on each other, so it's important that I be very clear about the starting point. I've created the folder c:/mygame, and inside I have these folders and files:

/common
/creator
/starter.fps
DeleteDSOs.bat
DeletePrefs.bat
glu2d3d.dll
main.cs
MyGame.exe (This is my renamed Torque Demo App)
OpenAL32.dll
opengl2d3d.dll
wrap_oal.dll

The /common, /creator, and /starter.fps folders are straight copies from the TGE 1.4 HEAD (March 08, 2006) "example" folder. If you are unsure how to download the HEAD from CVS, please see this excellent tutorial.

MyGame.exe, opengl2d3d.dll and glu2d3d.dll are all fresh builds from the HEAD. If you are unsure how to create a new Visual C++ Project from the Torque SDK, please see this tutorial.
All other files in the project root are just copied over from the "example" folder. The main.cs at the root currently has this line of code at the top: $defaultGame = "starter.fps";

Run the DeleteDSOs.bat and DeletePrefs.bat once each and you should be at the same starting place I was at when I started this whole thing.


First Goal:
-----------
Our first objective will be to merge and eliminate the /common folder by moving all the subfolders in /common out to the project root (/client, /server or /creator). You should be able to go through these tutorials with any of the "mods" (like starter.racing, tutorial.base or demo), but I haven't tried any of these yet -- so, you'll probably have to make in-flight adjustments if you try anything other than starter.fps.

In the end, I'd like to see a directory structure like this:

/client
/creator
/server


Conventions:
------------

Throughout this tutorial, I'll be asking you to test your "Torque Demo" application (I use the name "mygame.exe" in the text below). It is important that you really do test your demo app at each of these points -- we'll be making a lot of changes and trying to make them all at once without testing at each CHECKPOINT would be a debugging nightmare.

I will use backslashes to show the path, from project root, to a C++ source code file. I'm showing the full path so that you can be absolutely sure that you are editing the correct file.

I will use forward slashes to show the path, from your game folder root (c:/mygame in my case), to a Torque Script file.

Phase 1: Merging GUI Files
--------------------------

1. Create a new folder at the game folder root named /client

2. Create a new folder under this new /client folder called /ui

3. Create a new folder at the project root named /server

4. Move /common/ui/*.gui to /client/ui

5. Update the path for FrameOverlayGui.gui in /common/client/metrics.cs to this:

exec("client/ui/FrameOverlayGui.gui");

6. Update the paths for all the rest of the *.gui files in /common/client/canvas.cs from this:

exec("~/ui/ColorPickerDlg.gui");

to this:

exec("client/ui/ColorPickerDlg.gui");

7. Repeat step #6 for the other 10 exec statements in /common/client/canvas.cs

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App (c:/mygame/mygame.exe in my case) to make sure everything worked.


Next we'll work on the PNG files. I know these are going to be hard because I know for a fact that the source code has hard-coded paths in it to some of these files.


Phase 2: Merging PNG Files
--------------------------


1. Move /common/ui/folder.png and /common/ui/folder_closed.png to /creator/ui

2. Make a copy of /creator/ui/folder.png and rename it "bs.png"

3. Edit guiDirectoryTreeCtrl.cc and change this:

icons = StringTable->insert("common/ui/bs:common/ui/folder:common/ui/folder_closed");

to this:

icons = StringTable->insert("creator/ui/bs:creator/ui/folder:creator/ui/folder_closed");

4. Move the following files from /common/ui to /creator/ui:

audio.png
bs.png <--- I created this file in /common/ui to eliminate a console warning message
camera.png
default.png
fxfoliage.png
fxlight.png
fxshapereplicator.png
fxsunlight.png
hidden.png
interior.png
lightning.png
mission_area.png
particle.png
path.png
pathmarker.png
physical_area.png
precipitation.png
shape.png
shll_icon_passworded.png
shll_icon_passworded_hi.png
simgroup.png
simgroup_closed.png
simgroup_selected.png
simgroup_selected_closed.png
sky.png
static_shape.png
sun.png
terrain.png
trigger.png
water.png

5. Update the paths for each of these files in guiTreeViewCtrl.cc (in function GuiTreeViewCtrl::buildIconTable) starting around line 960.

6. Save, clean and build your C++ project

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.

For anything that we are moving out of /common to the /creator folder, you'll want to get into the World Editor Inspector in the game, and expand the top MissionGroup-SimGroup to make sure you can still see all the icons.


Phase 3: Merging More PNG Files
-------------------------------

1. Move the following files from /common/ui to /client/ui

CUR_3darrow.png
CUR_leftright.png
CUR_move.png
CUR_nesw.png
CUR_nwse.png
CUR_textedit.png
CUR_updown.png

2. Find the line "//-------------------------------------- Cursors" in /common/ui/defaultProfiles.cs (right around line 595) and update the paths for each GuiCursor below that line from "./CUR*" to "client/ui/CUR*" -- here's an example:

Change:

bitmapName = "./CUR_3darrow";

to:

bitmapName = "client/ui/CUR_3darrow";

3. Repeat the step above for each "CUR_*" file in /common/ui

4. Delete the files listed below in /common/ui (they aren't referenced in script or in source code):

NOTE: Be careful here, don't delete _all_ the "shll_*" files, only the ones listed below :-)

shll_bar_act.png
shll_bar_rol.png
shll_icon_dedicated.png
shll_icon_dedicated_hi.png
shll_icon_default.png
shll_icon_favorite.png
shll_icon_favorite_hi.png
shll_icon_notqueried.png
shll_icon_notqueried_hi.png
shll_icon_penguin.png
shll_icon_querying.png
shll_icon_querying_hi.png
shll_icon_timedout.png
shll_icon_tourney.png
shll_icon_tourney_hi.png
shll_sortarrow.png

5. Move /common/ui/shll_treeView.png to /creator/ui

6. In /common/ui/defaultProfiles.cs, find (right around line 301):

bitmap = "./shll_treeView";

and change to this:

bitmap = "creator/ui/shll_treeView";

NOTE: there will be 2 occurrences, make sure to change them both


7. In guiInspector.cc, find this line:

delButt->setField("Bitmap", "common/ui/inspector_delete");

...and change to:

delButt->setField("Bitmap", "creator/ui/inspector_delete");

8. Move these files from /common/ui to /creator/ui:

inspector_delete_d.png
inspector_delete_h.png
inspector_delete_i.png
inspector_delete_n.png

9. Save, clean and build your C++ project

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.


Phase 4: Merging Even More PNG Files
------------------------------------

IMPORTANT! These next 10 steps are done as separate steps because finding all lines with "./dark" will not also find all lines with "./osx" and vice-versa and the same goes for lines with "./torque". You need to search for each term separately. Also, if these next few changes are not all done up to the point where you can run mygame.exe again (or are done incorrectly), the game will run as normally up to the point where your avatar actually enters the sim. If the engine can't find even 1 of these files in even 1 reference in even 1 script, it will crash at this point. If this happens, go back over these steps again and make sure you've found absolutely every reference to the files being moved.


1. Find all paths starting with "./dark" in /common/ui/defaultProfiles.cs and change to the "." in the path to "client/ui"

2. Move these files from /common/ui to /client/ui:

darkBorder.png
darkScroll.png
darkTab.png
darkTabPage.png
darkWindow.png

3. Find all paths starting with "./osx" in /common/ui/defaultProfiles.cs and change to the "." in the path to "client/ui"

4. Move these files from /common/ui to /client/ui:

osxCheck.png
osxMenu.png
osxRadio.png
osxScroll.png
osxWindow.png

5. Find all paths starting with "./torque" in /common/ui/defaultProfiles.cs and change to the "." in the path to "client/ui"

6. Move these files from /common/ui to /client/ui:

torqueCheck.png
torqueMenu.png
torquePane.png
torqueRadio.png

7. Edit /starter.fps/client/ui/defaultGameProfiles.cs, change:

bitmap = "common/ui/darkScroll";

to this:

bitmap = "client/ui/darkScroll";

NOTE: there will be 2 occurrences, make sure to change them both


8. Copy /client/ui/darkScroll.png and /client/ui/osxScroll.png to /creator/ui/

9. Edit /creator/ui/creatorProfiles.cs, change:

bitmap = ($platform $= "macos") ? "common/ui/osxScroll" : "common/ui/darkScroll";

to this:

bitmap = ($platform $= "macos") ? "creator/ui/osxScroll" : "creator/ui/darkScroll";

10. Edit /creator/ui/creatorProfiles.cs again, and change:

bitmap = "common/ui/darkScroll";

to this:

bitmap = "creator/ui/darkScroll";

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.

Just a note, this is the first file I've seen so far in "common" that's actually referenced by more than 1 script between creator, client and starter.fps. In this case, the *Scroll.png files are used by both the client and the creator. So, the total "bulk" we've added to the code so far is 10k for 2 duplicated files -- not too bad for getting rid of practically the whole /common/ui folder.

Phase 5: defaultProfiles.cs and canvas.cs
-----------------------------------------

1. In the file /common/ui/defaultProfiles.cs change:

$Gui::fontCacheDirectory = expandFilename("./cache");
$Gui::clipboardFile      = expandFilename("./cache/clipboard.gui");

to this:

$Gui::fontCacheDirectory = expandFilename("client/ui/cache");
$Gui::clipboardFile      = expandFilename("client/ui/cache/clipboard.gui");

2. In the folder, /common/ui, delete these files:

icon.png
server_tabs.png


3. Move /common/ui/defaultProfiles.cs to /client/ui

4. In /common/client/canvas.cs, change:

exec("common/ui/defaultProfiles.cs");

to this:

exec("client/ui/defaultProfiles.cs");

5. Delete the folder "ui" in /common

6. In /client/ui/defaultProfiles.cs, find every occurence of "client/ui" in path names and change it back to "." Here's an example, change:

Gui::fontCacheDirectory = expandFilename("client/ui/cache");

to this:

Gui::fontCacheDirectory = expandFilename("./cache");

7. Repeat the previous step for all paths starting with "client/ui" in /client/ui/defaultProfiles.cs. This will switch the paths back to relative paths instead of absolute paths.


CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.


Congratulations!!! Those were the most difficult changes of all the changes we'll be doing. Believe it or not, the script changes are much simpler.

Next, merging the scripts...

Phase 6: Client and Server Scripts
-----------------------------------


1. Move the following files from /common/client to /client:

actionMap.cs
audio.cs
canvas.cs
message.cs
mission.cs
missionDownload.cs

2. Move the following files from /common/server to /server:

audio.cs
clientConnection.cs
commands.cs
game.cs
kickban.cs
message.cs
missionDownload.cs
missionInfo.cs
missionLoad.cs
server.cs

3. In /common/main.cs, change:

function initCommon()
{
   // All mods need the random seed set
   setRandomSeed();

   // Very basic functions used by everyone
   exec("./client/canvas.cs");
   exec("./client/audio.cs");
}

function initBaseClient()
{
   // Base client functionality
   exec("./client/message.cs");
   exec("./client/mission.cs");
   exec("./client/missionDownload.cs");
   exec("./client/actionMap.cs");
}

to this:

function initCommon()
{
   // All mods need the random seed set
   setRandomSeed();

   // Very basic functions used by everyone
   exec("client/canvas.cs");
   exec("client/audio.cs");
}

function initBaseClient()
{
   // Base client functionality
   exec("client/message.cs");
   exec("client/mission.cs");
   exec("client/missionDownload.cs");
   exec("client/actionMap.cs");

}

4. Also in /common/main.cs, change:

function initBaseServer()
{
   // Base server functionality
   exec("./server/audio.cs");
   exec("./server/server.cs");
   exec("./server/message.cs");
   exec("./server/commands.cs");
   exec("./server/missionInfo.cs");
   exec("./server/missionLoad.cs");
   exec("./server/missionDownload.cs");
   exec("./server/clientConnection.cs");
   exec("./server/kickban.cs");
   exec("./server/game.cs");
}

to this:

function initBaseServer()
{
   // Base server functionality
   exec("server/audio.cs");
   exec("server/server.cs");
   exec("server/message.cs");
   exec("server/commands.cs");
   exec("server/missionInfo.cs");
   exec("server/missionLoad.cs");
   exec("server/missionDownload.cs");
   exec("server/clientConnection.cs");
   exec("server/kickban.cs");
   exec("server/game.cs");
}

5. Also in /common/main.cs, change:

function onExit()
{
   echo("Exporting client prefs");
   export("$pref::*", "./client/prefs.cs", False);

   echo("Exporting server prefs");
   export("$Pref::Server::*", "./server/prefs.cs", False);
   BanList::Export("./server/banlist.cs");

   OpenALShutdown();
   Parent::onExit();
}

to this:

function onExit()
{
   echo("Exporting client prefs");
   export("$pref::*", "client/prefs.cs", False);

   echo("Exporting server prefs");
   export("$Pref::Server::*", "server/prefs.cs", False);

   echo("Exporting server banlist");
   BanList::Export("server/banlist.cs");

   OpenALShutdown();
   Parent::onExit();
}

6. Move the following files in /common/client to /client (the paths in canvas.cs are already correct):

cursor.cs
help.cs
messageBox.cs
metrics.cs
recordings.cs
screenshot.cs

7. Delete the file "scriptDoc.cs" in /common/client (The correct location for this file is /creator/scripts, and there's already a copy of it there)

8. Delete the folder "server" in /common

9. Delete the folder "client" in /common


CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.


Congratulations! You just merged all the scripts (except for main.cs) out of /common !!! There's really not that much left to do -- you're almost done :-)


Phase 7: Misc Files
--------------------

1. Create a new folder called "help" under /client/ui

2. Move all the files in /common/help to /client/ui/help

3. Delete the folder /common/help

4. Move all *.hfl files in /starter.fps/client/ui to /client/ui/help (there should be 3 of them)

5. Move the folder "lighting" and all contents from /common to /client

6. Move the file /common/TestMap.jpg to the folder /creator/data

7. In \engine\interior\interior.cc (right around line 398), change this:

mWhite        = new TextureHandle("common/lighting/whiteAlpha255", MeshTexture);
   mWhiteRGB     = new TextureHandle("common/lighting/whiteNoAlpha", MeshTexture);
   mLightFalloff = new TextureHandle("common/lighting/lightFalloffMono", BitmapTexture, true);

to this:

mWhite        = new TextureHandle("client/lighting/whiteAlpha255", MeshTexture);
   mWhiteRGB     = new TextureHandle("client/lighting/whiteNoAlpha", MeshTexture);
   mLightFalloff = new TextureHandle("client/lighting/lightFalloffMono", BitmapTexture, true);

8. In \engine\interior\InteriorMap.cc (right around line 205), change this:

mWhite = new TextureHandle("common/lighting/whiteAlpha255", MeshTexture);

to this:

mWhite = new TextureHandle("client/lighting/whiteAlpha255", MeshTexture);

9. In \engine\terrain\terrData.cc (right around line 873), change this:

mDynLightTexture = TextureHandle("common/lighting/lightFalloffMono", BitmapTexture, true);

to this:

mDynLightTexture = TextureHandle("client/lighting/lightFalloffMono", BitmapTexture, true);

10. In \engine\game\fx\fxSunLight.cc (right around line 1420), change this:

mLocalFlareTextureName		= StringTable->insert("common/lighting/corona");
	mRemoteFlareTextureName		= StringTable->insert("common/lighting/corona");

to this:

mLocalFlareTextureName		= StringTable->insert("client/lighting/corona");
	mRemoteFlareTextureName		= StringTable->insert("client/lighting/corona");

11. In \engine\gui\utility\guiTransitionCtrl.cc (right around line 66), change this:

mTestTexture = new TextureHandle("common/TestMap", BitmapTexture);

to this:

mTestTexture = new TextureHandle("creator/data/TestMap", BitmapTexture);

12. Save, clean, and build your C++ Project

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.

Important! The file locations for the sun flares will need to be re-mapped in the mission file. To do this, hit F11 to get into the Editor, and then F3 for the World Editor Inspector. In the main MissionGroup Simgroup, edit the two paths under the "Media" section of the sunflare1 and sunflare2 objects, replacing "common" in the path with "client" and you'll be able to see the sun again. Move some other object slightly so the engine thinks you've made a change to the mission and then save the mission.

Note: At this point, it's a good idea (but not 100% necessary) to delete the *.ml files in your mission directory. This will force the engine to relight your mission.

Now, all we have left to do is merge the /common/main.cs

Phase 8: /common/main.cs
-------------------------


The changes we're making here to main.cs are only temporary changes to remove /common/main.cs from the execution stream. These changes are not the best way of doing things, but they will keep your Torque Demo app functional and running normally. The next tutorial in this series will focus on removing the onStart() and packaging functionality -- further simplifying the script codebase. The next tutorial in the series will assume you've made these changes.

1. In /starter.fps/main.cs and /creator/main.cs comment out this line:

loadDir(common)";

2. At your game folder root, in main.cs, change:

onStart();
   echo("Engine initialized...");

to this:

setRandomSeed();
   exec("client/canvas.cs");
   exec("client/audio.cs");
   onStart();
   echo("Engine initialized...");

3. In /starter.fps/client/init.cs (right around line 44), change this:

// The common module provides basic client functionality
   initBaseClient();

to this:

// This was originally in /common/main.cs::initBaseClient(), now we're just running
   // the scripts directly instead of doing all the re-direction.
   exec("client/message.cs");
   exec("client/mission.cs");
   exec("client/missionDownload.cs");
   exec("client/actionMap.cs");

...and while we're here, (right around line 54), change this:

initCanvas("Torque Game Engine", true);

to this:

initCanvas("Whatever you want your game name to be", true);

4. In /starter.fps/server/init.cs (right around line 53), change this:

// The common module provides the basic server functionality
   initBaseServer();

to this:

// This was originally in /common/main.cs::initBaseServer(), now we're just running
   // the scripts directly instead of doing all the re-direction.
   exec("server/audio.cs");
   exec("server/server.cs");
   exec("server/message.cs");
   exec("server/commands.cs");
   exec("server/missionInfo.cs");
   exec("server/missionLoad.cs");
   exec("server/missionDownload.cs");
   exec("server/clientConnection.cs");
   exec("server/kickban.cs");
   exec("server/game.cs");

5. Delete the file /common/main.cs

6. Delete the folder /common

CHECKPOINT --> Run DeleteDSOs.bat and DeletePrefs.bat at your game folder root, then run your Torque Demo App to make sure everything worked.


CONGRATULATIONS!!!

My next tutorial in this series will build on this one, where I'll be showing you how to merge /starter.fps into the /client, /server and /creator folders. After that, I'll be demonstrating how to remove the package and module functionality from the script code base.

I would like to thank the following people who helped with this tutorial:

Benjamin Bradley -- For helping with EULA clarifications, and being an all-around nice guy.

David Dougher -- For painstakingly combing through the instructions, testing them, and identifying typos, mistakes, and just Better Ways of Doing Things... and for being an all-around nice guy.

Edit: Typos and clarifications