Torque3D (client-side) on Linux discussion

#1 Okay, long time ago, when I got dedicated build of T3D running on my Linux server, I've looked at what is wrong with the rest of the engine to make a fully-functional, with GFX layer on (OpenGL ofc), SFX, etc..

The good thing, that most of the problematic stuff is already done (there was some refactoring done in T3D by GG to fix a few major issues, incl Linux-related onces), so things like network and basic OS layer are totally fine now. And I believe SFX should be working (yes, even on dedicated build you can try to init the SFX, haven't done it yet by myself though).

What is missing is:

GFX layer.
Input handling and UI-releated part of the Engine/source/platformX86UNIX.
The basic code for this folder was initially taken from the TGE, and it was running fine on Linux.
So, all what needs to be done is to be sure it works with current generation of distros.

Actually, the problematic files are all files in Engine/source/platformX86UNIX folder which have ".client." in filename.

I don't have a lot of spare time, but will try to play with it during weekends... and don't blame me too much if I disappear for a few weeks :)

#2 Decided to build it :)

First three source files should be fixed:

Engine/source/gfx/gl/gfxGLCubemap.cpp:

-#include "gfx/bitmap/DDSFile.h"
+#include "gfx/bitmap/ddsFile.h"

Fixed case of the ddsFile.h

Engine/source/platformX86UNIX/x86UNIXWindow.client.cpp:

-#include "core/fileStream.h"
+#include "core/stream/fileStream.h"

Fixed include.

Tools/projectGenerator/classes/Torque3D.php:

-        if ( Generator::$platform == "360" || Generator::$platform == "ps3" )
+        if ( Generator::$platform == "360" || Generator::$platform == "ps3" || Generator::$platform == "linux" )

Telling project generator, that for linux platform it shouldn't use "SHARED" config.

Now, go to the buildFiles/config folder and copy the file:
Templates/Full/buildFiles/config/project.conf
to:
Templates/Full/buildFiles/config/project.linux.conf
(yes, removing old content)
Now edit this file and change:
Torque3D::beginConfig( "win32", "Full" );
to:
Torque3D::beginConfig( "linux", "Full" );
And don't forget to comment out the webDeploy, as we don't plan to use it atm :)

Generate projects and run:
make debug.
So far the problematic files are:

Engine/source/platformX86UNIX/x86UNIXInput.client.cpp
Engine/source/platformX86UNIX/x86UNIXInputManager.client.cpp
Engine/source/platformX86UNIX/x86UNIXWindow.client.cpp
Engine/source/platformX86UNIX/x86UNIXFont.client.cpp
Engine/source/platformX86UNIX/x86UNIXGL.client.cpp
Engine/source/platformX86UNIX/x86UNIXOpenAL.client.cpp
Engine/source/platformX86UNIX/x86UNIXOGLVideo.client.cpp

Everything else compiles fine.
Soon will push a new branch to github with all changes applied.

#3 Changes pushed to linux-client branch. Feel free to grab it!

#4 an essential part.
thanks for you and all other's time and effort on it.i always wanted to start with linux side developing.but i do not have any knowledge or idea on linux platform.
so waiting for a minimum workable version to start work on linux platform.

#5 Probably want to keep an eye on the stuff going on over here with OpenGL development.

#6 Not crucial, but the LightInfoExType issue could probably be resolved slightly more cleanly by making an empty default constructor which doesn't add the light type to the type list, then using the normal name-based constructor in the initializer.

e.g.

// In "LightInfoExType":
LightInfoExType() {;}

// In code
LightInfoExType ShadowMapParams::Type;

#7 Great! Downloading right now and will try to compile on Ubuntu soon. ( Should be merged with the CE someday )

#8 @James: this one looks good! Thanks, will try this.

@All:
After looking a bit deeper, there are a couple more issues:

1. we are totally missing "windowManager", which should be under Engine/source/windowManager/linux/ folder. So this one should be implemented from the zero.
2. Engine/source/gfx/gl/ggl/x11/ needs to be updated/implemented.

#9 I didn't yet have a look, but as the Windows version uses QT, it should be possible to do alike for the windowmanager. GL stuff should be very similar to the Community edition:
https://github.com/T3DCE/CE-OSLab
Thanks

#10 Torque3D doesn't use QT at all.

The QT libraries are there only because Torque Toolbox is built on top of QT.

#11 @fyodor

3. platform/menus (Native menu bars & popup menus) - AFAIK these don't exist in SDL. In this case it is probably best to use a GuiMenuBar & GuiPopUpMenuCtrl instead, just like TGE/TGEA.

#12 @James: I've never worked with such on Linux, so can't even say anything.. Probably you are right on this, but anyway that will require a lot of work.

#13 SDL doesn't provide good native widgets at all. It seems like the sanest thing to do (long term) would actually be to rewrite it with a QT, GTK+ or Wx based UI. What do we rely on SDL for, besides providing the OpenGL context?

#14 @ Fyodor: Generate projects and run:
make debug.

hmm how? Cannot find any makefile...
Thanks

#15 Instructions on how to generate makefiles are in first post of this pull request:

cd Templates/Full
./generateProjects.command
cd buildFiles/Make
make debug

or for dedicated:

cd Templates/Full
./generateProjects.command
cd buildFiles/Make_Ded
make clean

#16 anyone try the linux branch recently? (not the linux-client)
I know the default MIT branch of T3D definitely does not build on linux out of the box and imagine that is why there is a linux branch.

a clean linux box attempting to build the dedicated server runs into this error
../../../../Engine/lib/libvorbis/lib/registry.h:28:1: error: unknown type name ‘vorbis_func_floor’
../../../../Engine/lib/libvorbis/lib/registry.h:29:1: error: unknown type name ‘vorbis_func_residue’
../../../../Engine/lib/libvorbis/lib/registry.h:30:1: error: unknown type name ‘vorbis_func_mapping’
make: *** [Debug/libvorbis/libvorbis/lib/registry.h.o] Error 1

Just followed the steps to create the build files then ran make clean and make debug and get the error.

#17 k a little digging and realized the files didnt merge completely. When you try to build the default MIT T3D version you might see this error as well.

Resync'd and seems to work fine.just make sure its actually the linux branch, github has a tendency to give you the main branch instead. doh.

#18 This probably isnt the best thread for this, but I know some of you are working with linux dedicated server so maybe this will help. I dont know if it has been fixed since i last merged in changes from the T3D dev branch for linux dedicated server but there is a bug with the Forest object and NULL GFX device. The actual bug has to do with imposters which the forest object uses. If you look in ts/tsShapeInstance.cpp in the function void TSShapeInstance::buildInstanceData(TSShape *_shape, bool loadMaterials) at the very end you will see:

if (loadMaterials && mShapeResource) 
    mShape->setupBillboardDetails( mShapeResource.getPath().getFullPath() );

The issue is that setupBillboardDetails should not run on a null GFX device (and the other time it is ran, there is a check for that)

So to fix it just change the code to this:

f (loadMaterials && mShapeResource && GFXDevice::devicePresent()) 
    mShape->setupBillboardDetails( mShapeResource.getPath().getFullPath() );

-Tim

#19 @Tim:
I checked in your fix into the development branch as issue 155. Thanks!

However, I'm wondering if the forest should be loading materials (and therefore building billboards) at all on the server. This seems like a waste to me. I've created issue 157 to look into this at a later time. If you had any comments on this, it would probably be best to put them in the issue rather than in the Linux thread here.

Thanks!

- Dave

#20 @David, Yeah it shouldn't be needed. I have been working on our linux dedicated server over the past week or so to get it ready for general release so anyone can run a dedicated server for our game. Our game currently is around 350mb and I was able to get the dedicated server down to 13mb by removing all of the textures, sounds, and gui/font files. Everything still works fine so I would assume materials are not needed anywhere on the server. (although I do still have the .cs material files in there so I am not positive if they are needed or not but I am guessing not)

One other change i made was to remove SDL from the linux dedicated server. The reason I had to look into this is Ubuntu 12.04 and 12.10 64bit have a package manager bug that will not let you install SDL 32bit binaries. So looking into it, there was some code in 2 files that didn't have #ifndef TORQUE_DEDICATED and I had to rewrite the semaphore code in x86UnixSemaphore.cpp. I replaced it with standard unix semaphore handling which I am pretty sure is what SDL was using underneath anyway. Over all it was pretty simple and let me remove the SDL dependency (there is no reason a dedicated server should need SDL anyway)

I can give you those changes if you want to commit them as well if you think they would fit well in the dev branch.