Eval() not aware of *all* variables in scope

#1 A quick scan of the C++ code shows that eval()-okay variables are those created or used within the code-block to which it belongs. Presumably this means that all variables referenced between the closest surrounding open-curly-brace, {, and the function call are fair game. But nothing in the function call itself? That's unexpected to say the least.

#2 An interesting results. This should be a bug.

#3 I copied your code and tried it in my latest build of TGB and I got the correct results

4
5
Done!

I'll try and test this against our latest release and see if I'm experiencing the same problem or if it was somehow fixed in the latest release.

#4 Not sure if this will help at all but I have the MacOSX version of 1.13.

By the way, the function I defined above is written in a script file and the function is called from the console (by hitting "~" in-game).

#5 I tried it in windows version 1.13 and it worked fine for me as well. I just copied Eric's code into game.cs ran it brought up the console and ran the function. I got back

4
5
Done!

Maybe something with how the Mac handles memory?

#6 Okay, this is weird:

I tested in the MacOSX TGB 1.12 and the function returned the expected results.

Looks like it's an issue with TGB 1.13 for MacOSX.

Can someone else verify this on a Mac?

#7 I have this issue on TGB 1.1.3 ver. (win)
I put Eric's code to the game.cs (an empty project named TGB, after install)at the end and get the output:

... 
Loading compiled script TGB/managed/brushes.cs.
Loading compiled script TGB/gui/mainScreen.gui.
Compiling TGB/gameScripts/game.cs...
Loading compiled script TGB/gameScripts/game.cs.
4

Done!
Executing TGB/gameScripts/game.cs.
4
5
Done!
Setting screen mode to 800x600x32 (w)...
Executing TGB/data/levels/testgame.t2d.
...

The first time - start TGB
The second one - run a game

#8 Ignoring the fact that this may or may not be a bug with eval, the easy fix is don't use eval:

//Test Function:
function evalSucksDontUseItUnlessYouAbsolutelyHaveTo(%x, %y)
{
     call(echo, %x);
     call("ec" @ "ho", %y);

     echo("Done!");
}

// Test Code:
evalSucksDontUseItUnlessYouAbsolutelyHaveTo(4, 5);

For objects:

%foo = new SimObject();
%foo.call("dump");

%obj.call() was a fairly recent addition, but as far as I remember it was done before 1.1.3 shipped. The normal form of call has existed pretty much since the beginning, I think.

Using eval() to call functions means that Torque has to compile the script code you pass to eval every time you call eval(). Then it has to actually interpret the code before it gets to call the function. This takes, comparatively, a lot of time. This takes only slightly less time then putting the string you pass to eval() in a separate script file and executing it. As there is no DSO cache for eval(), that slight difference may actually get negated by the full recompile on every call of eval().

Using call()/%obj.call() the code is compiled once when the script is compiled and calling the function will be only slightly slower then calling that function directly. Essentially, the overhead is slightly less then two script function calls. This is significantly less overhead then if eval was used.

Other examples off the top of my head:

// Indirect object naming
for(%i = 0;%i < 10;%i++)
{
   // I don't remember if the brackets are required for the object name expression
   new SimObject(%foo @ %i);
   
   (%foo @ %i).dump();
}


// Hybrid Function/Object Method callbacks
function doCallback(%callback, %arg1, %arg2, %arg3)
{
   // Note that call()/%obj.call() can take any number of arguments, the only
   // required one is the first which is the name of the function. The rest
   // are passed to the called function.
   
   %obj = getField(%callback, 0);
   if(! isObject(%obj))
      call(%callback, %arg1, %arg2, %arg3);
   else
      %obj.call(getField(%callback, 1), %arg1, %arg2, %arg3);
}

// Do function callback
doCallback("fooFunc", "arg1");

// Do object callback
doCallback(%obj TAB "dump");

So, in summary:

1. Using eval() to call functions is extremely slow.
2. Using eval() to handle dynamic object names is extremely slow.
3. There are lots of other reasons eval() is bad that I haven't mentioned.
4. There are only very few cases where you absolutely need eval(), and those cases are the reason why eval() exists.
5. If you are affected by the bug that started this thread, then there is a high chance that you are using eval() when you shouldn't be using it.

End of rant ;)

T.

#9 A few things:

@Igor:

Quote:

[b]Loading compiled script TGB/gameScripts/game.cs.[/b]
4

Done!
[b]Executing TGB/gameScripts/game.cs.[/b]
4
5
Done!

Emphasis mine.

That seems to be similar to my situation. I found that if I went in and called exec() on the script file containing the function, it would work without issue, producing the expected output. If you run it from code that's simply been 'compiled', however, things don't seem to play nicely.

This lead me to believe that I had my exec() statements in the wrong area. Starting with TGB 1.1.3 I changed the way I program in two ways:
1) I actually use datablocks.
2) I moved my script exec() calls to the main.cs file (so that they are all exec()'d along with game.cs.

Due to the realization regarding exec() and eval() above, I tried moving the exec() statements to the game.cs file. This had one extremely negative drawback: all of my datablocks stopped getting recognized by the editor. ...Until I test ran the game. But by that point, all datablock references were broken/removed from the level file and forced me to re-add them. Further, any subsequent testing would add datablocks to TGB (after two runs, I'd have two of each datablock available to me in the Object Create rollout).

So here's a new question: where should files get exec()'d? Should we put all datablock / for-in-editor use scripts in the main.cs file and all game logic in game.cs? Is that a fair 'rule of thumb'? It is convenient to simply replay a level after editing some scripts (without restarting TGB)... though we'd still have to restart if main.cs-located files got edited. Is there potentially an issue with this?

@Tom:
Your point is well taken and highly appreciated. I did experiment in using call() to achieve dynamic callbacks (I had a hunch that call() was more efficient than eval()). The issue I have with it is that you are restricting your developer to a limited number of arguments. In the example above, you have limited any callback made with doCallback() to three arguments. The wonderful thing about eval() is that no such restriction exists. You can pass an entire string with as many parameters as you want.

Unless you know of a way to pass a string of parameters to call()... for the life of me I haven't been able to get call() to return more than a "parse error" or "Unknown command" (even with echo()) when I pass it a string list of arguments (obviously trying to de-stringify it before the call).

#10 You can fake variable arguments. Just specify a whole bunch of them in the function decl and when called any missing arguments will just get set to "". That point was actually what I was trying to illustrate by declaring 3 args and then never passing all of them. It is, however, kinda lame ;)

Just in case I'm not entirely following what you're saying or you're not entirely following:

call(echo, "hello");

function f(%a1, %a2, %a3)
{
   echo("a1 = " @ %a1);
   echo("a2 = " @ %a2);
   echo("a3 = " @ %a3);
}

call(f, "1", "2", "3");
call(f, "1", "2");
call(f, "1");

In other words, call can take any number of arguments. It just gives the function you call all of them.

I was actually thinking a while ago about adding some form of varargs support to torque script, but the need was never great enough so I never got around to it. I may revisit it at some point, but it's unlikely I'll have time for a good long while.

I am really tired, sorry if this post didn't make sense.

T.

#11 @Tom: No, I gotcha. My point was that I really hope to avoid this:

function t2dSceneObject::generalFuncWitCallback(%this, %data1, %data2, %func, %a1, %a2, %a3, %a4, %a5, %a6, %a7, %a8, %a9, %a10)
{
     // Do stuff with data.  Maybe make some coffee.

     // Woops!  I'm done!  Here, let's call the general callback.  I hope 10 arguments was enough!
     call(%func, %a1, %a2, %a3, %a4, %a5, %a6, %a7, %a8, %a9, %a10);
}

See? In eval() world, we have:

function t2dSceneObject::generalFuncWitCallback(%this, %data1, %data2, %callback)
{
     // Do stuff with data.  Maybe make some coffee.

     // Woops!  I'm done!  Here, let's call the general callback.
     eval(%callback);
}

I'm currently working out the details on how to implement the following function (seems to work for the most part):

// Resizes an object's value linearly to a specified size over the specified amount of time.
// Adapted from my fade function, explained [url=http://www.garagegames.com/mg/forums/result.thread.php?qt=59356]here[/url].
// You can see where I'm getting bit by the bug in this function, too.
function SimObject::LinearChangeOverTime(%this, %getFunction, %setFunction, %toValue, %time, %command)
{
	if(%time > $FadeGranularity)
	{
		%currentVal = [i]eval(%this @ "." @ %getFunction @ "();");[/i]
		%updatesRemaining = %time / $FadeGranularity;
		%currentVal += (%toValue - %currentVal) / %updatesRemaining;		

		[i]eval(%this @ "." @ %setFunction @ "(" @ %currentVal @ ");");[/i]
		%this.schedule($FadeGranularity, LinearChangeOverTime, %getFunction, %setFunction, %toValue, %time - $FadeGranularity, %command);
	}
	else
	{
		[i]eval(%this @ "." @ %setFunction @ "(" @ %toValue @ ");");[/i]
		eval(%command);
	}
}

Now, certainly I see where I could, should, and will (heh), use call() instead of eval() (the italicized lines in the code block above) but that final line of the function is important. Certainly I could use a call-based version of the function, updating it every time I find that "Oops, I need more arguments at my disposal" but that makes the code a bit harder to read.

What I would like to know is if you can get n-arguments all into one string and do something like:
%this.call(%command, (%argList));

That would be clean, powerful, and efficienter.

Get some rest, man!

#12 TGB 1.1.3, MacOSX 10.4.9:

==>$foo = new SimObject();
==>$foo.call("dump");
<input> (0): Unknown command call.
  Object (2602) SimObject

It appears that %obj.call() didn't make it in time for 1.1.3.

How would one go about adding that call to 1.1.3....?

Aheh.

#13 Actually, this is Tom's mistake - there is no such method in SimObject. You can easily add this method though.

#14 As I said, %obj.call() was a fairly recent addition. I thought it had made it into 1.1.3 but I guess I was wrong on that :)

Here's the code:

ConsoleMethod( SimObject, call, const char*, 2, 0, "( %args ) - Dynamically call a method on an object." )
{
   argv[1] = argv[2];
   return Con::execute( object, argc - 1, argv + 1 );
}

@Eric,

The n-arguments thing is what I meant by varargs and is not currently supported. As I said (or intended to say, if I didnt), it's something I've been thinking about adding for a while but haven't gotten around to yet. Yeh, having lots of arguments is lame and irritating, but there are times when it's unavoidable. Have a look at how messageClient() and friends are implemented if you want a skin crawling experience ;) That is actually one of the things that made me want to do varargs in the first place.

The eval(%command) in your function looks like one of those fringe cases where there is little choice. The choice is basically to use something like callbacks or allow calling of arbitrary code as a result of the thing finishing. Arguably, this is a generalized function that shouldn't really know what it's used for and thus eval() is probably the appropriate solution. Also the eval() is only being called once as a result of the function finishing and thus is likely not much of a performance hit.

T.

Edit: I just remembered that %obj.call() may have at one time been called %obj.callMethod() ... don't know if that code ever shipped or even if it was in a GG codebase, I work on a lot of different codebases and they all blur into one after a while ;)

#15 @Tom: Awesome, thanks for the code snippet! That will make things a whole lot simpler!

The thing about the one remaining eval() callback in my function above is that it is, as you mentioned, supposed to be incredibly generic. I have left it alone in my fade() function as well. My rationalization is that I very frequently want something to happen once something's completely faded in or out and, depending on the object/time/situation/etc. that can change. Eval() in this case is wonderful because I can schedule almost anything based off of a fade, including chaining several events together, with very few lines of code.

I've tested the fade function with something around 5 objects simultaneously and haven't felt any hiccups (I'm running on a G4 1.67Ghz Powerbook with 1GB RAM). We'll see how that goes, though, as my game design calls for the potential of having oodles of objects fading at a time. So far, so good.

I really do appreciate the help and examples (I really, really liked the dynamic object naming example. That's something I've been looking for!). Good stuff!

As for the eval() issue that started this thread... any ideas? (Appears to have something to do with the way the code is exec()'d.)

#16 Hmm. Looking at the code, an eval() is identical to an exec() except the code comes from a string instead of a file.

Thus, the code executing in eval() should be in it's own scope. Which would mean the bug is that you can access local variables from the calling function at all, since they should be out of scope. Thus the correct fix would be to disallow access to local vars from the calling function altogether.

I'm not really sure what to make of that, thoughts ?

In either case, it is evident from the code that this is going to take more time to investigate then I currently have. I will keep an eye on this thread and see if I can look at it in a week or two.

T.

#17 To add %myObj.call() to TGB 1.1.3 and earlier, add the following (reprinted from above) to SimBase.cc at around line 666 (what it was for me... oooh, ominous):

ConsoleMethod( SimObject, call, const char*, 2, 0, "( %args ) - Dynamically call a method on an object." )
{
   argv[1] = argv[2];
   return Con::execute( object, argc - 1, argv + 1 );
}

The above is already included in TGB 1.5 Beta 3 (at least).