Why exactly is the package set up this way?
by Scott Bruno · in Torque Game Engine · 03/22/2002 (2:21 am) · 34 replies
Or more precisely, why is there an instance of the game object in the engine lib? I realize the startup and platform stuff is in the engine; that's not the problem.
If "Torque Lib" produces a .lib, and "Torque Example" produces an executable, then it seems to me that DemoGame and main.cc should be native to Example, not Lib.
In theory I should be able to construct my own .exe, link it with engine.lib, then provide and instantiate a GameInterface-derived object in my own code to get things going.
However, that's not the case here. "Torque Example" doesn't even link with the lib produced by "Torque Lib". Instead, the game is instantiated and run inside the code that makes up the lib. That's why the example app doesn't have any soure of its own, and is essentially just the engine codebase inserted into an .exe project.
To put it into perspective, this would be like inserting the Windows source files into your app and compiling it along with your own source, rather than just including the appropriate headers and linking with the appropriate libs.
At this point some of you are probably wondering what the big deal is. You can, after all, just copy and rename torqueDEMO.exe if your project requires only scripts and art assets in order to make it work.
But that doesn't cut it for me, or for some other folks, judging by some of the posts I've seen in these forums. If you plan to do something other than a Tribey shooter then you're going to be writing some C++. Those objects are going to need to be driven by the application heartbeat (aka "game loop", aka "frame tick"). But since DemoGame and DemoGame::main() exist in the engine, I have to alter the engine's source base in order to give my code access to the heartbeat. This strikes me as odd.
I'm not criticizing the engine at all, just its packaging. The appearance is that this package was put together in such a way as to make it instantly accessbile to non-programmers. This I can totally understand, but in the process it sort of broke things for people who are programmers.
Ideally, I shouldn't have to touch the engine source at all unless I want to add to or modify its core functionality. I shouldn't need to deal with the "Torque Lib" source until Garage Games updates it, at which time I rebuild the lib, deal with any interface changes that may exist, and re-link my project. In other words I should be able to keep the engine source pure. After all, that's why we put things into libs in the first place. But because "Torque Lib" is not a pure lib, I must now tinker with its source and be prepared to suffer potential heartache when it comes time to merge with whatever changes GG introduces to it in the future. Erm... I assume CVS does actually merge and not just replace...
Signing off.
If "Torque Lib" produces a .lib, and "Torque Example" produces an executable, then it seems to me that DemoGame and main.cc should be native to Example, not Lib.
In theory I should be able to construct my own .exe, link it with engine.lib, then provide and instantiate a GameInterface-derived object in my own code to get things going.
However, that's not the case here. "Torque Example" doesn't even link with the lib produced by "Torque Lib". Instead, the game is instantiated and run inside the code that makes up the lib. That's why the example app doesn't have any soure of its own, and is essentially just the engine codebase inserted into an .exe project.
To put it into perspective, this would be like inserting the Windows source files into your app and compiling it along with your own source, rather than just including the appropriate headers and linking with the appropriate libs.
At this point some of you are probably wondering what the big deal is. You can, after all, just copy and rename torqueDEMO.exe if your project requires only scripts and art assets in order to make it work.
But that doesn't cut it for me, or for some other folks, judging by some of the posts I've seen in these forums. If you plan to do something other than a Tribey shooter then you're going to be writing some C++. Those objects are going to need to be driven by the application heartbeat (aka "game loop", aka "frame tick"). But since DemoGame and DemoGame::main() exist in the engine, I have to alter the engine's source base in order to give my code access to the heartbeat. This strikes me as odd.
I'm not criticizing the engine at all, just its packaging. The appearance is that this package was put together in such a way as to make it instantly accessbile to non-programmers. This I can totally understand, but in the process it sort of broke things for people who are programmers.
Ideally, I shouldn't have to touch the engine source at all unless I want to add to or modify its core functionality. I shouldn't need to deal with the "Torque Lib" source until Garage Games updates it, at which time I rebuild the lib, deal with any interface changes that may exist, and re-link my project. In other words I should be able to keep the engine source pure. After all, that's why we put things into libs in the first place. But because "Torque Lib" is not a pure lib, I must now tinker with its source and be prepared to suffer potential heartache when it comes time to merge with whatever changes GG introduces to it in the future. Erm... I assume CVS does actually merge and not just replace...
Signing off.
#2
Tim Gift acknowledged this as being important... though, didn't agree that there should be a short term goal of getting the engine into a linkable form...
Unless people understand and are vocal about it... this will probably not happen anytime soon... and that is a real drag...
From my .plan ...
Preface: Torque is very powerful and extremely hard work has gone into it. The problem presented here has NOTHING to do with the way the engine is designed... it has to do with how it is linked...
An important news bulletin: Everything that uses Torque code in your project has to be contained within the same module... you can't link from another module to it... currently this means sticking the entire Torque source tree into the exe for your project... this also means that any of your code that uses Torque must reside here too... and since this is what everybody has to do... it means cramming their code in as well...
This is bad-> You CANNOT call a Torque function or derive a Torque class outside of the module the Torque source resides in... to use plain english: royal suckage
Sharing the tool's static library is out because the game can't use one... a static library also doesn't fix the linkage problem of using the code in more than one place:
Illegal:
My.dll linked to static Torque.lib
My.exe linked to My.dll AND static Torque.lib
Furthermore, there is NO NEED for the static library,or the seperate compilation step it necessitates:
Dynamic linking allows people from multiple projects to use Torque in a consistent manner... we can use Torque consistently in our OWN projects and subprojects(tools)... we are not limited in any way WHERE we can use Torque functionality... AND code can be more readily SHARED... as LIBRARIES and _not_ zip files of sources to MERGE INTO your (and Torque's) source tree.
In addition, there are some nice side effects: other development environments and languages being opened to use Torque via the dll. Loading and unloading of code for plugin support, Distribution of new code without a link step to an existing application... etc.
I realize that there is more work to be done and probably some hard changes in store for the API... this doesn't preempt dynamic linking... in fact, dynamic linking is just a step in the process of making Torque shine even brighter
There isn't much work to do: simply adding export declations which can be turned on or off with a define! So if anyone is convinced dynamic linking buys them nothing; they can go on with having the entire engine build down at the lowest level: the application.exe
"Torque on top!"
-Joshua Ritter
This thread has a more indepth though diluted discussion of the topic:
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=4801
-J
05/13/2002 (7:19 pm)
I made a similar post some weeks ago... Tim Gift acknowledged this as being important... though, didn't agree that there should be a short term goal of getting the engine into a linkable form...
Unless people understand and are vocal about it... this will probably not happen anytime soon... and that is a real drag...
From my .plan ...
Preface: Torque is very powerful and extremely hard work has gone into it. The problem presented here has NOTHING to do with the way the engine is designed... it has to do with how it is linked...
An important news bulletin: Everything that uses Torque code in your project has to be contained within the same module... you can't link from another module to it... currently this means sticking the entire Torque source tree into the exe for your project... this also means that any of your code that uses Torque must reside here too... and since this is what everybody has to do... it means cramming their code in as well...
This is bad-> You CANNOT call a Torque function or derive a Torque class outside of the module the Torque source resides in... to use plain english: royal suckage
Sharing the tool's static library is out because the game can't use one... a static library also doesn't fix the linkage problem of using the code in more than one place:
Illegal:
My.dll linked to static Torque.lib
My.exe linked to My.dll AND static Torque.lib
Furthermore, there is NO NEED for the static library,or the seperate compilation step it necessitates:
Dynamic linking allows people from multiple projects to use Torque in a consistent manner... we can use Torque consistently in our OWN projects and subprojects(tools)... we are not limited in any way WHERE we can use Torque functionality... AND code can be more readily SHARED... as LIBRARIES and _not_ zip files of sources to MERGE INTO your (and Torque's) source tree.
In addition, there are some nice side effects: other development environments and languages being opened to use Torque via the dll. Loading and unloading of code for plugin support, Distribution of new code without a link step to an existing application... etc.
I realize that there is more work to be done and probably some hard changes in store for the API... this doesn't preempt dynamic linking... in fact, dynamic linking is just a step in the process of making Torque shine even brighter
There isn't much work to do: simply adding export declations which can be turned on or off with a define! So if anyone is convinced dynamic linking buys them nothing; they can go on with having the entire engine build down at the lowest level: the application.exe
"Torque on top!"
-Joshua Ritter
This thread has a more indepth though diluted discussion of the topic:
www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=4801
-J
#3
It only costs 100 bucks, don't expect the Unreal/Quake3/LithTech engine.
05/13/2002 (7:54 pm)
The way I see it, you get what you pay for.It only costs 100 bucks, don't expect the Unreal/Quake3/LithTech engine.
#4
It only costs 100 bucks, don't expect the Unreal/Quake3/LithTech engine."
The cost isn't the point. If the engine sucked then you could say sure, you get what you pay for. But it doesn't suck, and more importantly, the point of the post is not about whether it sucks. The point is about how it's put together.
05/14/2002 (7:25 pm)
"The way I see it, you get what you pay for. It only costs 100 bucks, don't expect the Unreal/Quake3/LithTech engine."
The cost isn't the point. If the engine sucked then you could say sure, you get what you pay for. But it doesn't suck, and more importantly, the point of the post is not about whether it sucks. The point is about how it's put together.
#5
Brings to mind that old saw: you can either run with the big dogs, or stand on the porch and bark.
05/14/2002 (7:59 pm)
Instead of seeing a problem, those of you who would rather the engine be organized or structured in a different way might consider this an opportunity. A golden one, at that.Brings to mind that old saw: you can either run with the big dogs, or stand on the porch and bark.
#6
This isn't a matter of opinion... the layout is poor... the question is what path the engine takes to dynamic linking... and when...
I don't think Tim Gift would mind me quoting him from an email:
"Having to include all the source directly into your project is very lame, but the solution here is to be build a torque core library :) I believe a library solves many of the issues you bring up and is something that should be worked on"
I agree with this, though I think that getting the engine linkable should happen ASAP... with any hard changes to the API and code seperation happening later and when feasible...
-J
05/14/2002 (8:14 pm)
A golden opportunity to maintain a version against the HEAD... and to think I almost missed it!This isn't a matter of opinion... the layout is poor... the question is what path the engine takes to dynamic linking... and when...
I don't think Tim Gift would mind me quoting him from an email:
"Having to include all the source directly into your project is very lame, but the solution here is to be build a torque core library :) I believe a library solves many of the issues you bring up and is something that should be worked on"
I agree with this, though I think that getting the engine linkable should happen ASAP... with any hard changes to the API and code seperation happening later and when feasible...
-J
#7
05/14/2002 (8:20 pm)
A golden opportunity to boost your royalty rate...
#8
According to GG, the Torque API isn't ready for being built into a linkable module... which I disagree with... link now... better API when possible...
Ultimately the Torque community doesn't seem to "get it" or care... so blah... I am not in this to maintain my own unique little Torque variant...
This has just proven frustrating... some day Torque will be linkable... you'll see...
-J
05/15/2002 (1:26 am)
I don't think it is wise to go ahead and do here... otherwise, I would have already... and not for some hope of a higher royaltyAccording to GG, the Torque API isn't ready for being built into a linkable module... which I disagree with... link now... better API when possible...
Ultimately the Torque community doesn't seem to "get it" or care... so blah... I am not in this to maintain my own unique little Torque variant...
This has just proven frustrating... some day Torque will be linkable... you'll see...
-J
#9
I'd be happy with a static lib, but otherwise, I'm ok with how it is now (for now).
Isnt this just an excuse not to get on with some work? there's always going to be one problem or another. Hell, we all know its not perfect, but you can either get on with it, or wait forever until it meets every requirement.
Of course asking for change is alright, but I doubt anyone's going to change it for you. There are bigger fish to fry.
Phil.
05/15/2002 (1:38 am)
Personally I had DLL's, you can forget the DLL idea because its not cross platform.I'd be happy with a static lib, but otherwise, I'm ok with how it is now (for now).
Isnt this just an excuse not to get on with some work? there's always going to be one problem or another. Hell, we all know its not perfect, but you can either get on with it, or wait forever until it meets every requirement.
Of course asking for change is alright, but I doubt anyone's going to change it for you. There are bigger fish to fry.
Phil.
#10
A static library does not meet the requirements here... notice that the static library THAT ALREADY exists isn't used by the Torque "Demo"...
Static libraries pretty much blow in comparison... dynamic linking was invented for a reason.. mostly to solve static linking's shortcomings... there are many sources of information as to the differences between the two... any why dynamic linking is preferable...
Frankly, I don't think anything is going to happen here anytime soon... as you said, there are bigger fish to fry.. or so it seems...
I think linking is a pretty basic and big fish... I volunteered to do the work within the understanding of GG... but as there wasn't an agreement that this should happen in the near term, and I am not interested in my own mutant Torque, well... it didn't happen...
This doesn't mean that when the subject comes up... I won't shout, scream, and flail about wildly :)
-J
05/15/2002 (1:44 am)
Linux and Mac don't have dynamic linking? I was pretty darned sure they did? What do they have? Certainly the OS isn't statically linked? *wink*A static library does not meet the requirements here... notice that the static library THAT ALREADY exists isn't used by the Torque "Demo"...
Static libraries pretty much blow in comparison... dynamic linking was invented for a reason.. mostly to solve static linking's shortcomings... there are many sources of information as to the differences between the two... any why dynamic linking is preferable...
Frankly, I don't think anything is going to happen here anytime soon... as you said, there are bigger fish to fry.. or so it seems...
I think linking is a pretty basic and big fish... I volunteered to do the work within the understanding of GG... but as there wasn't an agreement that this should happen in the near term, and I am not interested in my own mutant Torque, well... it didn't happen...
This doesn't mean that when the subject comes up... I won't shout, scream, and flail about wildly :)
-J
#11
-Tim aka Spock
05/15/2002 (2:12 am)
Here is a question Id like to know the answer to. If you put the engine in dlls and created your game app totally from your code, could you release your games source code or would it violate the license agreement. similiar to what id does with quake 3 and RTCW. Release your game code, not the engine's code. If so it could dramatically increase the modding options of your game.-Tim aka Spock
#12
This is just one example of what being bound to the engine source code precludes...
I am curious on Mac and Linux... Dynamic linking is done via .so files under Linux? Is this right?
-J
05/15/2002 (2:16 am)
Yes... you would have to code hooks into the exe for the game code to use... but you certainly could do this... This is just one example of what being bound to the engine source code precludes...
I am curious on Mac and Linux... Dynamic linking is done via .so files under Linux? Is this right?
-J
#14
05/15/2002 (7:26 am)
if you are worried about supporting mods then don't that is what the scripting engine is for. I just subclass whatever object you want to be moddable and put script assesable functions to dynamically alter the behavior of the object. Use the datablock only for initialization.
#15
05/15/2002 (8:20 am)
well i think you could also add all your game specific stuff into a dll... much like quake works... then release the src code to the dll
#16
Personally I don't like the dll idea, it opens the engine up to HACKS and CRACKS way to easily!
Every Half-Life cheat works by wedging itself into between the engine and the mod .dll, the cheats are nothing more than pass thru mods themselves that alter the information between engine and game .dll.
This is why every Quake derived game mod has rampant cheating!
Now I know that having a monolithic code base is less maintainable and that NOTHING is hack proof, it is much more difficult to go and dissassemble an executable and hack it and recompile it than it is to write a proxy mod for cheats, it is also EASIER to detect any monkeying around with a monolithic .exe than detecting wedge type cheats.
05/15/2002 (8:25 am)
NO DLL's!!!!!!Personally I don't like the dll idea, it opens the engine up to HACKS and CRACKS way to easily!
Every Half-Life cheat works by wedging itself into between the engine and the mod .dll, the cheats are nothing more than pass thru mods themselves that alter the information between engine and game .dll.
This is why every Quake derived game mod has rampant cheating!
Now I know that having a monolithic code base is less maintainable and that NOTHING is hack proof, it is much more difficult to go and dissassemble an executable and hack it and recompile it than it is to write a proxy mod for cheats, it is also EASIER to detect any monkeying around with a monolithic .exe than detecting wedge type cheats.
#17
"Isnt this just an excuse not to get on with some work?"
As the original poster I'll assume the responsibility of taking this one.
Work, you say? Torque is strictly for my personal use. My work goes on the shelves (as I see some of yours has) and is getting on quite well, thanks. Now I'm not sure where the tone of your response came from, but it's uncalled for and totally inapplicable, so I suggest you put it back.
That having been said, I'd be happy with a static lib, too. Is it necessary? No. Working with an engine that's under development is not uncommon. Generally what you'll do is pick a release that seems to have everything you need and build on that. Perhaps that's just the case here for anyone who's doing deep work on the source side of things rather than scripting their game into existence.
05/15/2002 (9:36 am)
According to Phil Carlisle:"Isnt this just an excuse not to get on with some work?"
As the original poster I'll assume the responsibility of taking this one.
Work, you say? Torque is strictly for my personal use. My work goes on the shelves (as I see some of yours has) and is getting on quite well, thanks. Now I'm not sure where the tone of your response came from, but it's uncalled for and totally inapplicable, so I suggest you put it back.
That having been said, I'd be happy with a static lib, too. Is it necessary? No. Working with an engine that's under development is not uncommon. Generally what you'll do is pick a release that seems to have everything you need and build on that. Perhaps that's just the case here for anyone who's doing deep work on the source side of things rather than scripting their game into existence.
#18
I tend to leave the word "YOU" as directed at another user out of any response... this helps things stay on target... and really, a vague off remark is less disruptive then a direct assault...
For the people that think DLL's are some windows(tm) voodoo.. the .dll part is just a file extension for dynamic linked libraries... your favorite OS has it too... under Linux for example, it's a .so
I think this topic gets mildly heated because people don't really understand it... but hey... what do I know :)
-J
05/15/2002 (9:41 am)
The tone of an email or post is always much worse than it really is... I tend to leave the word "YOU" as directed at another user out of any response... this helps things stay on target... and really, a vague off remark is less disruptive then a direct assault...
For the people that think DLL's are some windows(tm) voodoo.. the .dll part is just a file extension for dynamic linked libraries... your favorite OS has it too... under Linux for example, it's a .so
I think this topic gets mildly heated because people don't really understand it... but hey... what do I know :)
-J
#19
Compatibility problems also come into play when you have a C++ engine like Torque. If I were to compile Torque into a DLL right now, anyone else's project that wanted to share the use of that DLL would have to use the same compiler due to the name mangling of exported C++ classes. Getting around this generally requires a COM-style approach, which would probably be quite an undertaking for a code base the size of Torque's.
Honestly, I've worked with projects of various size before, and I don't see why dynamic linking is such a killer app. The fact is nobody needs community support to do the work. Do the work, show everyone how it works and the advantages, and if it's popular I'm sure it can affect the official source layout. That's the way it works around here. People didn't salivate in anticipation when Melv May posted until he started showing people the awesome stuff he could produce.
It took me minutes to separate the projects in a text editor and get them correctly compiled. Getting them to run properly is a matter of restructuring some of the engine, but I'm not going to waste my time with what I believe to be a lateral move at this point. So, by all means, do it and prove yourself right. Otherwise go play with Crystal Space, they use that kind of modular approach and it's already done for you.
05/15/2002 (5:56 pm)
The mechanism for creating and using shared libraries differs between platforms. IIRC, in Windows, the library's code is linked right in with the executable, and the library file is locked. The UNIX way actually copies the library code into the executable's memory space. This is why you can upgrade shared libraries in Linux without needing to shut everything down and restart.Compatibility problems also come into play when you have a C++ engine like Torque. If I were to compile Torque into a DLL right now, anyone else's project that wanted to share the use of that DLL would have to use the same compiler due to the name mangling of exported C++ classes. Getting around this generally requires a COM-style approach, which would probably be quite an undertaking for a code base the size of Torque's.
Honestly, I've worked with projects of various size before, and I don't see why dynamic linking is such a killer app. The fact is nobody needs community support to do the work. Do the work, show everyone how it works and the advantages, and if it's popular I'm sure it can affect the official source layout. That's the way it works around here. People didn't salivate in anticipation when Melv May posted until he started showing people the awesome stuff he could produce.
It took me minutes to separate the projects in a text editor and get them correctly compiled. Getting them to run properly is a matter of restructuring some of the engine, but I'm not going to waste my time with what I believe to be a lateral move at this point. So, by all means, do it and prove yourself right. Otherwise go play with Crystal Space, they use that kind of modular approach and it's already done for you.
#20
Sharing functionality as a prebuilt dynamic library isn't central to the goal... though, it is a nice side effect... this isn't a replacement to compiling the engine...
If I was a hobbyist, I would be dicking around with this just to do it... well, I am not... nor am I trying to make anyone salivate...
I don't need some artificial challenge... yes, it can be done... no, it's not hard.. But it would be a maintenance nightmare against the CVS...
I am pretty much feeling like a masochist on the subject...
I don't think this will be adopted anytime soon... so ultimately: *BLAH*
-J
05/15/2002 (6:03 pm)
The underlying mechanism the OS uses for dynamic linking isn't of much interestSharing functionality as a prebuilt dynamic library isn't central to the goal... though, it is a nice side effect... this isn't a replacement to compiling the engine...
If I was a hobbyist, I would be dicking around with this just to do it... well, I am not... nor am I trying to make anyone salivate...
I don't need some artificial challenge... yes, it can be done... no, it's not hard.. But it would be a maintenance nightmare against the CVS...
I am pretty much feeling like a masochist on the subject...
I don't think this will be adopted anytime soon... so ultimately: *BLAH*
-J
Torque Owner Jarrod Roberson
I was wondering this also?