Game Development Community

Compiling torque on x86-64 (amd64) non functional?

by Antonio "Tony" Farinetti · in Torque Game Engine · 05/15/2004 (1:57 pm) · 109 replies

After getting the CVS like usual and executing this command:

make -f mk/configure.mk OS=LINUX COMPILER=GCC2 BUILD=DEBUG

and getting this back:

Current Configuration: this config is VALID
         OS: LINUX
   COMPILER: GCC2
      BUILD: DEBUG
    DIR.OBJ: out
 
To change the current configuration type:
 
make -f mk/configure.mk {arguments, ...}
 
required arguments:
  OS={WIN32 BEOS LINUX OpenBSD FreeBSD Solaris}
  COMPILER={CW6 VC6 GCC2 GCC3}
  BUILD={DEBUG RELEASE}
 
optional arguments:
  DIR.OBJ={path to store intermediate obj files}
 
Note: all arguments are case sensitive.

And then issuing make, the compile stops fast with:

tony@tantalus torque $ make
--> Compiling lpng/png.c
cc1: error: CPU you selected does not support x86-64 instruction set
cc1: error: CPU you selected does not support x86-64 instruction set
make[1]: *** [out.GCC2.DEBUG/lpng/png.obj] Error 1
make: *** [default] Error 2

Any ideas on how to fix this?

Thanks!
#61
04/01/2005 (10:38 am)
Paul email me at the address in my profile
#62
04/02/2005 (7:30 am)
@Benoit and Tony: patch and HOWTO sent to you both.
#63
04/02/2005 (7:51 am)
.
#64
04/02/2005 (12:01 pm)
Hi Thc,
There are really two threads here and it's a bit confusing to follow them both :] All interested parties here are running 64-bit linux already: 64-bit kernel, 64-bit libraries and 64-bit gcc. We want to get two different things working though:

1: Torque as a native 64-bit library. Benoit has sent me his patches so far and I'm probably going to work with him on making the whole codebase 64-bit clean. It looks like a lot of work though. Still, there will be fame, TV spots and jobs from GarageGames in the future if we get it right :]

2: Torque running as a 32-bit library but compiled by 64-bit gcc under a 64-bit kernel. I have this working right here: 64-bit kernel and 64-bit gcc produces a perfectly working 32-bit build of Torque if you pass it the right flags. The only problem is installing the additional 32-bit libraries you need for it to link against.
#65
04/02/2005 (12:26 pm)
:) Paul there isn't any confusion

Any women with that fame :)
#66
04/02/2005 (2:46 pm)
Paul & Benoit:

If you need someone to rewrite something to your specifications, contact me, I am capable :)
#67
04/03/2005 (10:31 am)
64-bit compilation successful!
64-bit execution of torquedemo_DEBUG.bin dies horribly at console initialisation!

I wasn't surprised at this - I think my offset patch is suspect. So now the next step is to thoroughly understand how the console system works and find out how I can get the Offset() macro working properly. I really hope that some simple #ifdefs will sort it out rather than having to make the entire codebase agnostic about the size of ints....

@Tony: prepare to be called upon :]

@Benoit: I made some minor changes to x86_64UNIXFileio.cc to reference the U64 member correctly so it would compile.
#68
04/03/2005 (4:33 pm)
Yeah i had noticed after sending the patch i had forgotten to revert the FileTime struct back to a variable (as it was) among other broken aspects. I have also switched to debugging with -O0 instead of -O2 and it segfaults even earlier in the init phase. Same for removing inlining. Also with -O2 ddd/gdb has a hard following whats going on.

As for the offset issue i have tried every variation i can think of and it doesn't seem to change anything.

I noticed in 1.3.1 and 1.4.0 that they are changing the Offset macro ot one of 3 possible combo's. Will try to see what happens there.
#69
04/04/2005 (1:13 pm)
I've done a bit of reading on offsets, casts and macros and found something that looks like it does the Right Thing no matter the width of the processor. Here's some example code:

#include
using namespace std;
#define Offset(T,f) ((size_t)(&(reinterpret_cast(__alignof__(T *)))->f) - __alignof__(T *))

struct Point {
int *x,*y;
};

int main(void)
{
Point a_point;
cout << Offset(Point, y) << endl;
return 0;
}

64-bit compilation:
$ g++ -o offset offsetof.cpp
$ ./offset
8

32-bit compilation:
$ g++ -o offset -m32 offsetof.cpp
$ ./offset
4

I am going to try this out in the source tree tomorrow.
#70
04/04/2005 (1:39 pm)
Er, that should be Offset(f, T) not Offset (T,f) :]
#71
04/04/2005 (8:40 pm)
Did some more testing using offsetof (stddef.h) also produces the same results.

Heres an upadte of the test with some of the other variations i had tried:

#include <iostream>
using namespace std;

#define Offset0(T,f) ((size_t)(&(reinterpret_cast<T*>(__alignof__(T *)))->f) - __alignof__(T *))
#define Offset1(T,m) ((int)(&((T *)1)->m) - 1)
#define Offset2(cls, x) ((size_t)((const char *)&(((cls *)0)->x)-(const char *)0))
#define Offset3(s_name, s_member) (__offsetof__ (reinterpret_cast <size_t> (&reinterpret_cast <const volatile char &> (static_cast<s_name *> (0)->s_member))))
#define Offset4(T,m) ((long)(&((T *)1)->m) - 1)

struct Point
{
    int  *w,*x;
    long *y,*z;
};

struct Area
{
    const int  *w,*x;
    const long *y,*z;
};

int main(void)
{
    Point a_point;
    Area  b_area;

    cout << "-----" << endl;
    cout << "Point" << endl;
    cout << "-0---" << endl;
    cout << Offset0(Point, w) << endl;
    cout << Offset0(Point, x) << endl;
    cout << Offset0(Point, y) << endl;
    cout << Offset0(Point, z) << endl;
    cout << "-1---" << endl;
    cout << Offset1(Point, w) << endl;
    cout << Offset1(Point, x) << endl;
    cout << Offset1(Point, y) << endl;
    cout << Offset1(Point, z) << endl;
    cout << "-2---" << endl;
    cout << Offset2(Point, w) << endl;
    cout << Offset2(Point, x) << endl;
    cout << Offset2(Point, y) << endl;
    cout << Offset2(Point, z) << endl;
    cout << "-3---" << endl;
    cout << Offset3(Point, w) << endl;
    cout << Offset3(Point, x) << endl;
    cout << Offset3(Point, y) << endl;
    cout << Offset3(Point, z) << endl;
    cout << "-4---" << endl;
    cout << Offset4(Point, w) << endl;
    cout << Offset4(Point, x) << endl;
    cout << Offset4(Point, y) << endl;
    cout << Offset4(Point, z) << endl;
    cout << "-5---" << endl;
    cout << offsetof(Point, w) << endl;
    cout << offsetof(Point, x) << endl;
    cout << offsetof(Point, y) << endl;
    cout << offsetof(Point, z) << endl;
    cout << "-----" << endl;
    cout << "Area" << endl;
    cout << "-0---" << endl;
    cout << Offset0(Area, w) << endl;
    cout << Offset0(Area, x) << endl;
    cout << Offset0(Area, y) << endl;
    cout << Offset0(Area, z) << endl;
    cout << "-1---" << endl;
    cout << Offset1(Area, w) << endl;
    cout << Offset1(Area, x) << endl;
    cout << Offset1(Area, y) << endl;
    cout << Offset1(Area, z) << endl;
    cout << "-2---" << endl;
    cout << Offset2(Area, w) << endl;
    cout << Offset2(Area, x) << endl;
    cout << Offset2(Area, y) << endl;
    cout << Offset2(Area, z) << endl;
    cout << "-3---" << endl;
    cout << Offset3(Area, w) << endl;
    cout << Offset3(Area, x) << endl;
    cout << Offset3(Area, y) << endl;
    cout << Offset3(Area, z) << endl;
    cout << "-4---" << endl;
    cout << Offset4(Area, w) << endl;
    cout << Offset4(Area, x) << endl;
    cout << Offset4(Area, y) << endl;
    cout << Offset4(Area, z) << endl;
    cout << "-5---" << endl;
    cout << offsetof(Area, w) << endl;
    cout << offsetof(Area, x) << endl;
    cout << offsetof(Area, y) << endl;
    cout << offsetof(Area, z) << endl;
    cout << "-----" << endl;

    return 0;
}
#72
04/06/2005 (12:43 pm)
This is a pretty enlightening thread, and I've added it to my list of threads to review. Good work, guys! Keep it up! :)
#73
04/06/2005 (1:40 pm)
Glad you're finding it useful Ben - perhaps you can help us out here a bit since you're a Torque maintenance, documentation and c++ guru :)

I'm currently using this definition of Offset in engine/console/consoleTypes.h :
Offset(T,m) ((long)(&((T *)1)->m) - 1)

and I understand how and why it works as it does. The only difference between it and the 32-bit version is it casts to a long instead of an int. So, no problems there. But it doesn't seem to do the right thing for all code when doing a 64-bit compile. For instance the audioDataBlock classes in engine/audio/audioDataBlock have the usual:

static void initPersistFields()

helper function declaration for adding member variables to the global string table (at least I think that's what it does :)). In the implementation of initPersistFields, there are calls like this one:

addField("useRoom", TypeBool, Offset(mUseRoom, AudioEnvironment));

All of the audio*.cc files compile without trouble. However, when I get down to the engine/gui/gui* classes I hit errors like this:

gui/guiControl.cc:106: warning: invalid access to non-static data member '
GuiControl::mConsoleCommand' of NULL object

That's when I did some reading and discovered that static void initPersistFields() should not even be able to access non-static member variables! (in ANSI C++ anyway). Now the gui code seems to be doing exactly the same thing as the audioDataBlock classes - using a static member function to add those fields to the string table. Which is where I'm stuck at the moment.

Questions:
Why does Offset even work inside a static member function?
What's the difference between the guiControl class and the audioEnvironment class (for example) that makes g++ bitch about the first but not the second?
Any ideas on what could be going wrong? I smell a pointer arithmetic bug here somewhere but I just can't see it.

Thanks for your time and trouble.
#74
04/06/2005 (3:42 pm)
Thanks Ben.

Paul do you have any links to share on that subject? Btw i will send you a patch to get the cpuinfo through parsing /proc/cpuinfo instead of the current defaults. it just needs some cleaning up first.

Edit: Paul, on Gentoo the gcc package is a straight no patch job or do they include any patches?

Edit2: Interesting thread here coding.derkeiler.com/Archive/C_CPP/comp.lang.cpp/2003-11/3069.html.
#75
04/07/2005 (12:22 pm)
Benoit: I've found the "static can't access non-static member variables" rule in a couple of places:
www.conestogac.on.ca/~atehrani/OOP/C_chap17.ppt
http://annwm.lbl.gov/~leggett/vars.html
and in my copy of Effective C++.

gcc on gentoo has a few patches - most are for arches other than x86 and amd64.

Thanks for the thread link - the one method proposed won't work because it means every class needs to use private inheritance from a struct, the other looks OK but might mean adding const offset variables for every member!

Ben, if you're also reading this, there's another thing that bothers me: why don't I get the same error message in 32-bit mode for the gui classes?

I'm going to compile the same file to asm in both 32 and 64-bit formats and see what the difference is.
#76
04/08/2005 (10:24 am)
Progress!

Here's the best way to define Offset on a 64-bit platform: #define Offset(T, m) ((long)(offsetof(m, T))

Easy eh? Yes it does compile with warnings but it works 100%. I've checked.

Now for the segfaults...
#77
04/08/2005 (10:51 am)
Hmmm now i've got to find a reason to leave early from work :)

Time go buy some redbull & coca-cola by the gallon :)
#78
04/08/2005 (6:07 pm)
So i could test it out i brought back -O2 in debug and now i am getting this during compilation

ts/tsShapeConstruct.cc:152: error: 'i' cannot appear in a constant-expression

Are you getting something similar? Or did you only include it with consoleTypes.h?

Edit: you can use this -Wno-invalid-offsetof to disable the warnings as i mentioned in this thread www.garagegames.com/mg/forums/result.thread.php?qt=21521.
#79
04/08/2005 (6:18 pm)
Erk - sorry, my definition above is the wrong way around. Here's a direct cut and paste from my engine/console/consoleTypes.h:

#ifdef __X86_64__
#define Offset(m,T) ((long)(offsetof(T, m))
#else
#define Offset(m,T) ((int)(&((T *)1)->m) - 1)
#endif

Don't forget to change it in engine/platform/event.h too.

I've left out all optimisations for the moment and have tracked down a problem with Torque's implementation of realloc . Further bulletins as events warrant...
#80
04/08/2005 (6:21 pm)
Thanks for the pointer on realloc will check it out.

Btw have you tried enabling DEBUG_GUARD? If so does it work for you?