SimObjectId vs. S32
by Aaron Wieland · in Torque Game Builder · 08/18/2005 (7:57 pm) · 10 replies
This probably falls into the category of Things Nobody But Me Would Ever Care About, but here goes...
I'm autogenerating (is that an oxymoron?) wrapper classes in Smalltalk to make it easier to interact with T2D. I'd like to improve the method/field wrappers so that the return values are automatically coerced from strings to numbers, booleans, SimObjects, etc. Fields that reference SimObjects have a type of SimObjectPtr, appropriately enough. But console methods that answer a SimObject id have a return type of S32, making them indistinguishable from methods that return an integer representing something completely different (such as a count). There's already a typedef for SimObjectId, but it's never used in console method headers. Why not?
I just noticed that SimObjectId is unsigned, so that might be one reason, if you need to sometimes answer a negative value to represent an error. But look at fxTileMap2D::getTileLayer() for example; it answers a SimObjectId, yet the return type for the console method that wraps it is S32.
So, for the sake of everyone who is generating wrappers (a.k.a. Jason Swearingen and me), you might want to consider changing this. ;-)
EDIT: I mentioned Jason's name because he's created wrappers for T2D.NET, but I don't know whether he actually cares about this issue.
I'm autogenerating (is that an oxymoron?) wrapper classes in Smalltalk to make it easier to interact with T2D. I'd like to improve the method/field wrappers so that the return values are automatically coerced from strings to numbers, booleans, SimObjects, etc. Fields that reference SimObjects have a type of SimObjectPtr, appropriately enough. But console methods that answer a SimObject id have a return type of S32, making them indistinguishable from methods that return an integer representing something completely different (such as a count). There's already a typedef for SimObjectId, but it's never used in console method headers. Why not?
I just noticed that SimObjectId is unsigned, so that might be one reason, if you need to sometimes answer a negative value to represent an error. But look at fxTileMap2D::getTileLayer() for example; it answers a SimObjectId, yet the return type for the console method that wraps it is S32.
So, for the sake of everyone who is generating wrappers (a.k.a. Jason Swearingen and me), you might want to consider changing this. ;-)
EDIT: I mentioned Jason's name because he's created wrappers for T2D.NET, but I don't know whether he actually cares about this issue.
#2
Am I missing something?
08/18/2005 (9:43 pm)
Oooooo... another straight up one//-----------------------------------------------------------------------------
// Get Tile Layer Count.
//-----------------------------------------------------------------------------
ConsoleMethod(fxTileMap2D, getTileLayerCount, S32, 2, 2, "Returns Tile Layer Count.")
{
// Get Tile Layer Count.
return object->getTileLayerCount();
}
// Get Tile Layer Count.
U32 fxTileMap2D::getTileLayerCount(void)
{
// Return Tile Layer Count.
return mTileLayerList.size();
};Am I missing something?
#3
In the Vector class:
So even the function itself is asking for trouble.
08/18/2005 (9:47 pm)
In the fxTileMap2D classVector<fxTileLayer2D*> mTileLayerList;
In the Vector class:
S32 size() const;
So even the function itself is asking for trouble.
#4
08/18/2005 (9:57 pm)
Good points. I had to add many questionable typecasts to the engine code before TBE would compile it without errors (still plenty of warnings, though).
#5
In C#, you never (i think i'm safe saying never... or at least ALMOST never) add types unless you have a specific reason for doing so.
That is one thing I hate looking at C++ code. in most complex projects are sooo many macros #defining stuff left and right, that if you dont have a IDE properly configured (or a tool such as Source Insight) then you are pretty much screwed trying to find the definitions for these.
I know there are real (and good) reasons for doing this, because on some systems a char is 8 bytes signed, on others it's 16 bytes unsigned, etc etc. it's a way of making your program agnostic to the compiler. In C#, you dont NEED to be compiler agnostic, because all compilers conform to the same standards, and a char == char regardless of if it's on Mono+X86+linux or MS+ia64+Vista
If you are talking about taking the results of .GetId() and passing it to another ConsoleFunction, well you have a point about how it's kind of a pain doing this. But since all types in torquescript are strings, and all consoleFunctions and consoleMethods take in strings, i figured that is is OK to force the user to pass in strings to all of my functions. an acceptable evil. In the future I plan on manually going through (or hiring someone to go through) each and every ConsoleFunction/Method and add overloads that take in the "real" types, but then of course I will just convert these back to strings the moment the function is entered, and pass those strings back to the 'original' string overload of the method.
@Philip: Yeah there are inconsistancies, with SimObjectID being S32 vs U32.. i havent run any of my simulations long enough to see what happens when i get to 10,000 SimObjects, let alone the 4billion you would need to see this problem manifest. Dont get me wrong, if there is a 4billion limit (or even 8 billion) limit of simobjects that can ever be created, that IS a problem.. i would assume that objectID's are recycled after a certain number, but i HOPE this is a perf area that Melv and GarageGames has a solution planned for.
08/18/2005 (10:42 pm)
@Aaron: I'm honestly not too sure what you are looking for. If you are asking that SimObjectID's be used as the type instead of S32 for functions like GetId then i actually disagree with you.In C#, you never (i think i'm safe saying never... or at least ALMOST never) add types unless you have a specific reason for doing so.
That is one thing I hate looking at C++ code. in most complex projects are sooo many macros #defining stuff left and right, that if you dont have a IDE properly configured (or a tool such as Source Insight) then you are pretty much screwed trying to find the definitions for these.
I know there are real (and good) reasons for doing this, because on some systems a char is 8 bytes signed, on others it's 16 bytes unsigned, etc etc. it's a way of making your program agnostic to the compiler. In C#, you dont NEED to be compiler agnostic, because all compilers conform to the same standards, and a char == char regardless of if it's on Mono+X86+linux or MS+ia64+Vista
If you are talking about taking the results of .GetId() and passing it to another ConsoleFunction, well you have a point about how it's kind of a pain doing this. But since all types in torquescript are strings, and all consoleFunctions and consoleMethods take in strings, i figured that is is OK to force the user to pass in strings to all of my functions. an acceptable evil. In the future I plan on manually going through (or hiring someone to go through) each and every ConsoleFunction/Method and add overloads that take in the "real" types, but then of course I will just convert these back to strings the moment the function is entered, and pass those strings back to the 'original' string overload of the method.
@Philip: Yeah there are inconsistancies, with SimObjectID being S32 vs U32.. i havent run any of my simulations long enough to see what happens when i get to 10,000 SimObjects, let alone the 4billion you would need to see this problem manifest. Dont get me wrong, if there is a 4billion limit (or even 8 billion) limit of simobjects that can ever be created, that IS a problem.. i would assume that objectID's are recycled after a certain number, but i HOPE this is a perf area that Melv and GarageGames has a solution planned for.
#6
The situation you describe isn't an issue because my code automatically coerces all arguments to strings by sending the message #asT2DParameter; a String answers itself, a SimObject answers its ID as a string, etc. It's a very simple and robust solution.
08/19/2005 (12:42 am)
Quote:If you are talking about taking the results of .GetId() and passing it to another ConsoleFunction, well you have a point about how it's kind of a pain doing this.Actually, I'm talking about going in the opposite direction. There are ConsoleMethods that return the ID, and I'd like my wrapper methods to answer the actual object.
The situation you describe isn't an issue because my code automatically coerces all arguments to strings by sending the message #asT2DParameter; a String answers itself, a SimObject answers its ID as a string, etc. It's a very simple and robust solution.
#7
are you using my model for wrapping as a template? or are you doing your differently? (like marshaling the entire object)
08/19/2005 (2:33 am)
Ahh, well that's similar to what i'm doing, but yeah, i just output the int. up to the dev to figure out what they want to do with it :)are you using my model for wrapping as a template? or are you doing your differently? (like marshaling the entire object)
#8
According to a quick test, it seems that performance for method calls using your approach is better by an order of magnitude. I was passing a reference to the SimObject in both cases -- never the ID -- but I suspect that the time needed to look up the object by ID is negligible (at least I hope so, considering how often it's done while executing script). So, if performance becomes an issue, I'll try your method.
Another difference is that you generated the wrappers by parsing the C++ source directly (I think), and I parsed the output of writeOutClasses() and writeOutFunctions(). In hindsight, that was a mistake; the usage strings are unreliable, and I can't know exactly how many arguments a method or function takes. I'll probably eliminate the parsing entirely, and instead use the engine's API for reflecting on the scripting classes; e.g., AbstractClassRep::getClassList(), Namespace::getEntryList().
08/19/2005 (2:27 pm)
I was already working on my wrappers when I downloaded T2D.NET. I've browsed your code a couple of times to see how you did it. My implementation is more similar to TGEPython; I use the scripting interface (so the return value is always a string), whereas you call the C++ functions directly. For example, Con::execute(SimObject* object, S32 argc, const char *argv[]) is used to execute console methods. I don't really need the wrappers, but they make the syntax a lot nicer, plus they allow better use of the tools in the IDE -- to find senders and implementors, detect mispelled message sends, etc.According to a quick test, it seems that performance for method calls using your approach is better by an order of magnitude. I was passing a reference to the SimObject in both cases -- never the ID -- but I suspect that the time needed to look up the object by ID is negligible (at least I hope so, considering how often it's done while executing script). So, if performance becomes an issue, I'll try your method.
Another difference is that you generated the wrappers by parsing the C++ source directly (I think), and I parsed the output of writeOutClasses() and writeOutFunctions(). In hindsight, that was a mistake; the usage strings are unreliable, and I can't know exactly how many arguments a method or function takes. I'll probably eliminate the parsing entirely, and instead use the engine's API for reflecting on the scripting classes; e.g., AbstractClassRep::getClassList(), Namespace::getEntryList().
#9
As an aside, I imagine the performance of searching for object IDs to be negligable compared to the number of string conversions being done in the interface functions for TScript. If you can avoid directly using the TScript interface functions and instead calling the real thing yourself, you should.
08/19/2005 (4:54 pm)
Quote:According to a quick test, it seems that performance for method calls using your approach is better by an order of magnitude. I was passing a reference to the SimObject in both cases -- never the ID -- but I suspect that the time needed to look up the object by ID is negligible (at least I hope so, considering how often it's done while executing script). So, if performance becomes an issue, I'll try your method.
As an aside, I imagine the performance of searching for object IDs to be negligable compared to the number of string conversions being done in the interface functions for TScript. If you can avoid directly using the TScript interface functions and instead calling the real thing yourself, you should.
#10
On the other hand, I just ran a more realistic performance test, and the results were quite different. The first test was very naive; I compared ConsoleMethod(SimObject, getId), executed through the script interface, with SimObject::getId(), if that makes any sense. In other words, in the second case, I wasn't even calling the console method, but rather the method that it wraps; there was no need to marshal any parameters besides the object itself.
This time, I tested ConsoleMethod(fxSceneObject2D, setVisible) -- executed via the scripting interface versus calling cfxSceneObject2DsetVisible() (the corresponding macro-generated function, as T2D.NET does) directly. The script interface was only about 50% slower. Because setVisible() is a very simple method, the overhead of calling it accounts for most of the time.
Most of the time is probably spent on the Smalltalk side, preparing the arguments: (1) adding them to a collection and (2) converting that collection into something compatible with char**. I previously reduced the time by about 40% by optimizing (1) -- not by doing anything particularly clever, but simply by being less stupid. ;-) I'm sure that other optimizations are possible.
Speaking of stupidity, I'm not an expert in C++, and I haven't found a way of calling static functions from another file. Whenever I want to use a static function in my DLL interface (defined in its own file), I have to add a non-static wrapper function to the file that contains it. Is there a better way?
@Smaug:
When you refer to reducing the number of string conversions, are you talking about calling the console method or the code that it wraps (e.g., cSimObjectgetId() or SimObject::getId(), respectively)? Even if you call the C++ function corresponding to the console method instead of using Con::execute(), the arguments have to be strings. The return value doesn't, however.
08/19/2005 (10:36 pm)
I took another look at the ConsoleMethod macro, and realized that "self" is represented as a SimObject pointer, not an ID. So, my caveat doesn't apply.On the other hand, I just ran a more realistic performance test, and the results were quite different. The first test was very naive; I compared ConsoleMethod(SimObject, getId), executed through the script interface, with SimObject::getId(), if that makes any sense. In other words, in the second case, I wasn't even calling the console method, but rather the method that it wraps; there was no need to marshal any parameters besides the object itself.
This time, I tested ConsoleMethod(fxSceneObject2D, setVisible) -- executed via the scripting interface versus calling cfxSceneObject2DsetVisible() (the corresponding macro-generated function, as T2D.NET does) directly. The script interface was only about 50% slower. Because setVisible() is a very simple method, the overhead of calling it accounts for most of the time.
Most of the time is probably spent on the Smalltalk side, preparing the arguments: (1) adding them to a collection and (2) converting that collection into something compatible with char**. I previously reduced the time by about 40% by optimizing (1) -- not by doing anything particularly clever, but simply by being less stupid. ;-) I'm sure that other optimizations are possible.
Speaking of stupidity, I'm not an expert in C++, and I haven't found a way of calling static functions from another file. Whenever I want to use a static function in my DLL interface (defined in its own file), I have to add a non-static wrapper function to the file that contains it. Is there a better way?
@Smaug:
When you refer to reducing the number of string conversions, are you talking about calling the console method or the code that it wraps (e.g., cSimObjectgetId() or SimObject::getId(), respectively)? Even if you call the C++ function corresponding to the console method instead of using Con::execute(), the arguments have to be strings. The return value doesn't, however.
Torque 3D Owner MrPhil (Philip Ludington)
//----------------------------------------------------------------------------- // Get Tile Layer. //----------------------------------------------------------------------------- ConsoleMethod(fxTileMap2D, getTileLayer, S32, 3, 3, "(layerIndex) - Returns Tile Layer by Index.") { // Get Tile Layer. return object->getTileLayer( dAtoi(argv[2]) ); } // Get Tile Layer. SimObjectId fxTileMap2D::getTileLayer( U32 layerIndex ) { // Get Layer Count. U32 layerCount = getTileLayerCount(); // Check Layer Index. if ( layerIndex >= layerCount ) { // Warn. Con::warnf("fxTileMap2D::getTileLayer() - Invalid Tile Layer '%d'! Currently '%d' Layers available.", layerIndex, layerCount); return -1; } // Return Tile Layer ID. return mTileLayerList[layerIndex]->getId(); }From engine/console/simBase.h:
fxTileMap2D::createTileLayer and fxSceneGraph2D::getScenePhysicsMaxIterations are the same way.
Definitely looks like trouble to me.