Mind your SimSets
by Robert Blanchet Jr. · 07/22/2006 (2:43 am) · 19 comments
It has been a long time since I have updated my plan. In fact I haven't updated it since I was hired by GarageGames over two years ago, although it only seems like yesterday. Over the course of those two years I have been witness to lots of cool shit. Some you already know about: TGB, TSE, Marble Blast Xbox, Think Tanks Xbox, various incarnations of 360 dev kits, and Marble Blast Ultra just to name a few. And some things you don't know about, but when you do find out you'll find yourself smiling, nodding your head, and whispering 'Hey, that's some cool shit!'.
Anyway, that is not why I am posting today.
With the release of TGB behind us there has been an increased amount of discussion around SimSets and I thought I would chime in on that discussion. Now, before I begin I should probably mention I'm generally fairly poor at explaining myself, especially when it comes to topics of a technical nature and especially when it is late at night. I like to blame this on my brain wiring.. I just understand things differently that doesn't necessarily translate well to text or speech. For this reason I am using lots of script snippets below!
I am going to start off by asking a simple question. What is wrong with the following script:
script:
console output:
At first glance the above function seems fairly harmless. In fact the bug is so cleverly deceptive as to be mocking. For those of you who proudly shout 'it's a memory leak!' please give yourself a pat on the back and enjoy a cookie. As for the rest of you, for shame. I'm kidding, I'm kidding. Actually, the bug is rather sneaky and I've seen many people, new and skilled alike, fall victim to it including myself (but you can pretend I didn't say that).
So, lets look a little closer at the offending lines of code:
What we have here is a for loop that continues until %i is equal to or greater than the number of objects in the SimSet. Now, sometimes our brains fail to make the connection (SimSets are NOT arrays) and mistakingly believe that SimSet::getCount will return the number of elements initially in the SimSet each iteration of the loop, in this example 10. This is wrong because in the body of the loop we deleted an object thereby removing the object from the SimSet, thus decreasing the element count as a result.
Bonus round!
Multiple Choice: How many ScriptObjects did the above script fail to delete properly? Was it:
a) One
b) Five
c) All but one
Let's have a look, shall we? Below is 'programmer art', enjoy!
As you can see from the outline above, the script function failed to delete HALF of the objects in the SimSet. Thats a lot of objects to leave lurking around. Now imagine if that function is called multiple times in your game scripts, it adds up quickly! So, let's try and fix it.
script:
console output:
Uh, oh.. whats going on here?! Well to be truthful, I lied to you earlier. A SimSet doesn't look exactly like how I represented it in the 'programmer art' above. But I did it to drive home a point:
SimSets are NOT arrays
To help us understand what is going on, lets modify the example1 script function.
script:
console output:
SimSets do not guarantee any order. To be as effecient as possible, when an object is to be removed from a SimSet, they take the last element in the set and replace the element they are removing, and then remove the last element (this should be stated more clearly, but I fail). So what about that fix?
script:
The fix I like is to replace the for loop, with a while loop. Sometimes, however, it is not ideal to use a while loop and in those situations I recommend you pay extra attention. Take care that, if you call other methods from your for loops, that those functions don't delete the objects in the SimSet the loop is iterating over, and if they do it needs to be accounted for in the loop. In fact, the best practice would probably be to anticipate that some code, some where, will delete an object in your SimSet and to account for it in your loop appropriatly.
Anyway, that is not why I am posting today.
With the release of TGB behind us there has been an increased amount of discussion around SimSets and I thought I would chime in on that discussion. Now, before I begin I should probably mention I'm generally fairly poor at explaining myself, especially when it comes to topics of a technical nature and especially when it is late at night. I like to blame this on my brain wiring.. I just understand things differently that doesn't necessarily translate well to text or speech. For this reason I am using lots of script snippets below!
I am going to start off by asking a simple question. What is wrong with the following script:
script:
function example1()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
for(%i = 0; %i < %set.getCount(); %i++)
%set.getObject(%i).delete();
%set.delete();
}console output:
==>example1(); Object in SimSet: object0 Object in SimSet: object1 Object in SimSet: object2 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Object in SimSet: object7 Object in SimSet: object8 Object in SimSet: object9 Deleting Objects...
At first glance the above function seems fairly harmless. In fact the bug is so cleverly deceptive as to be mocking. For those of you who proudly shout 'it's a memory leak!' please give yourself a pat on the back and enjoy a cookie. As for the rest of you, for shame. I'm kidding, I'm kidding. Actually, the bug is rather sneaky and I've seen many people, new and skilled alike, fall victim to it including myself (but you can pretend I didn't say that).
So, lets look a little closer at the offending lines of code:
// I'm done, now cleanup
for(%i = 0; %i < %set.getCount(); %i++)What we have here is a for loop that continues until %i is equal to or greater than the number of objects in the SimSet. Now, sometimes our brains fail to make the connection (SimSets are NOT arrays) and mistakingly believe that SimSet::getCount will return the number of elements initially in the SimSet each iteration of the loop, in this example 10. This is wrong because in the body of the loop we deleted an object thereby removing the object from the SimSet, thus decreasing the element count as a result.
Bonus round!
Multiple Choice: How many ScriptObjects did the above script fail to delete properly? Was it:
a) One
b) Five
c) All but one
Let's have a look, shall we? Below is 'programmer art', enjoy!
SimSet::getCount:
v
Next element:
^
SimSet representation:
O0 O1 O2 O3 O4 O5 O6 O7 O8 O9
-----------------------------
iteration #0 (0 < 10 = true)
-----------------------------
v
O0 O1 O2 O3 O4 O5 O6 O7 O8 O9
^
iteration #1 (1 < 9 = true)
-----------------------------
v
[i]X[/i] O1 O2 O3 O4 O5 O6 O7 O8 [b]O9[/b]
^
iteration #2 (2 < 8 = true)
-----------------------------
v
[i]X[/i] [i]X[/i] O2 O3 O4 O5 O6 O7 [b]O8[/b] [b]O9[/b]
^
iteration #3 (3 < 7 = true)
-----------------------------
v
[i]X[/i] [i]X[/i] [i]X[/i] O3 O4 O5 O6 [b]O7[/b] [b]O8[/b] [b]O9[/b]
^
iteration #4 (4 < 6 = true)
-----------------------------
v
[i]X[/i] [i]X[/i] [i]X[/i] [i]X[/i] O4 O5 [b]O6[/b] [b]O7[/b] [b]O8[/b] [b]O9[/b]
^
iteration #5 (5 < 5 = false)
-----------------------------
v
[i]X[/i] [i]X[/i] [i]X[/i] [i]X[/i] [i]X[/i] [b]O5[/b] [b]O6[/b] [b]O7[/b] [b]O8[/b] [b]O9[/b]
^
Final representation:
-----------------------------
[b]O5[/b] [b]O6[/b] [b]O7[/b] [b]O8[/b] [b]O9[/b]As you can see from the outline above, the script function failed to delete HALF of the objects in the SimSet. Thats a lot of objects to leave lurking around. Now imagine if that function is called multiple times in your game scripts, it adds up quickly! So, let's try and fix it.
script:
function example2()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
%count = %set.getCount();
for(%i = 0; %i < %count; %i++)
%set.getObject(%i).delete();
%set.delete();
}console output:
==>example2(); Object in SimSet: object0 Object in SimSet: object1 Object in SimSet: object2 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Object in SimSet: object7 Object in SimSet: object8 Object in SimSet: object9 Deleting Objects... Set::getObject index out of range. (5) <input> (0): Unable to find object: '-1' attempting to call function 'Delete' Set::getObject index out of range. (6) <input> (0): Unable to find object: '-1' attempting to call function 'Delete' Set::getObject index out of range. (7) <input> (0): Unable to find object: '-1' attempting to call function 'Delete' Set::getObject index out of range. (8) <input> (0): Unable to find object: '-1' attempting to call function 'Delete' Set::getObject index out of range. (9) <input> (0): Unable to find object: '-1' attempting to call function 'Delete'
Uh, oh.. whats going on here?! Well to be truthful, I lied to you earlier. A SimSet doesn't look exactly like how I represented it in the 'programmer art' above. But I did it to drive home a point:
SimSets are NOT arrays
To help us understand what is going on, lets modify the example1 script function.
script:
function example3()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
for(%i = 0; %i < %set.getCount(); %i++)
{
echo("\nIteration #" @ %i);
%set.getObject(%i).delete();
for(%j = 0; %j < %set.getCount(); %j++)
{
%object = %set.getObject(%j);
echo(" Object in SimSet:" SPC %object.getName());
}
}
%set.delete();
}console output:
==>example3(); Object in SimSet: object0 Object in SimSet: object1 Object in SimSet: object2 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Object in SimSet: object7 Object in SimSet: object8 Object in SimSet: object9 Deleting Objects... Iteration #0 Object in SimSet: object9 Object in SimSet: object1 Object in SimSet: object2 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Object in SimSet: object7 Object in SimSet: object8 Iteration #1 Object in SimSet: object9 Object in SimSet: object8 Object in SimSet: object2 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Object in SimSet: object7 Iteration #2 Object in SimSet: object9 Object in SimSet: object8 Object in SimSet: object7 Object in SimSet: object3 Object in SimSet: object4 Object in SimSet: object5 Object in SimSet: object6 Iteration #3 Object in SimSet: object9 Object in SimSet: object8 Object in SimSet: object7 Object in SimSet: object6 Object in SimSet: object4 Object in SimSet: object5 Iteration #4 Object in SimSet: object9 Object in SimSet: object8 Object in SimSet: object7 Object in SimSet: object6 Object in SimSet: object5
SimSets do not guarantee any order. To be as effecient as possible, when an object is to be removed from a SimSet, they take the last element in the set and replace the element they are removing, and then remove the last element (this should be stated more clearly, but I fail). So what about that fix?
script:
function example4()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
while(%set.getCount())
%set.getObject(0).delete();
%set.delete();
}The fix I like is to replace the for loop, with a while loop. Sometimes, however, it is not ideal to use a while loop and in those situations I recommend you pay extra attention. Take care that, if you call other methods from your for loops, that those functions don't delete the objects in the SimSet the loop is iterating over, and if they do it needs to be accounted for in the loop. In fact, the best practice would probably be to anticipate that some code, some where, will delete an object in your SimSet and to account for it in your loop appropriatly.
About the author
#2
07/22/2006 (3:09 am)
Haha, what a classic misconception :) 
#3
07/22/2006 (6:35 am)
Been there, Done that. . . it is a good lesson to learn. . . ie use While to delete not For
#4
07/22/2006 (8:45 am)
couldnt you just do set.delete()? it was my understanding that deleting a simset automatically deletes its contents as well...
#5
I've been trying to track a sneaky little memory leak now for a month...
I think this may very well be it!
07/22/2006 (9:01 am)
I think I'm gonna cry...I've been trying to track a sneaky little memory leak now for a month...
I think this may very well be it!
#7
07/22/2006 (10:22 am)
@Sean: You might be thinking of a SimGroup. I think a SimGroup will delete the objects it contains. But SimSets don't touch the objects they contain when you delete the set because those objects might also be in other SimSets.
#8
07/22/2006 (10:47 am)
Well I think the real lesson to take away from this, is that you should use a SimGroup whenever possible. In most cases a SimGroup will make sense for your collection, unless the objects have to be owned by more than one set. Even when you need a SimSet for some reason, it usually makes sense to have a SimGroup somewhere. In that case, if the objects are contained in multiple SimSets, you won't run into issues figuring out at what point you have to delete the objects. You will know that the SimGroup owns the objects, and will be the one place where you will actually delete the objects.
#9
07/22/2006 (2:54 pm)
Domo arigato, Mr. Roberto, domo...domo 
#10
07/22/2006 (2:55 pm)
you can also iterate from back to front deleting objects, but that while loop is much prettier! 
#11
Lord knows I've fallen victim to this one :(
-J
07/23/2006 (3:48 pm)
Holy crap, I thought you died, Roberto :)Lord knows I've fallen victim to this one :(
-J
#12
calling a function to get a assumable static count into a for loop for every iteration is just wrong anyway.
If you want optimized code you don't do that sort of thing.
And as I think I've already discovered; Thera are no arrays in Torque Script... at all :P
Good info tho :)
Edit: Typos
07/23/2006 (5:11 pm)
In regards to the inital script example,calling a function to get a assumable static count into a for loop for every iteration is just wrong anyway.
If you want optimized code you don't do that sort of thing.
And as I think I've already discovered; Thera are no arrays in Torque Script... at all :P
Good info tho :)
Edit: Typos
#13
Now.. "Hey, that's some cool shit!".
07/23/2006 (8:35 pm)
This one got me not long ago as well. Neat Plan, useful, riddled with plain explanation, and funny...Now.. "Hey, that's some cool shit!".
#14
Actually, this will get you in a lot more trouble than you think. In general, you should default to using a SimSet, not a SimGroup unless you expressly need the "delete the group, delete all objects in set" functionality.
The reason is that if you accidentally place an object that is already in one SimGroup into another SimGroup, it will be silently removed from the first SimGroup.
Of course, it depends on what you need to do, but if you are simply using your SimXXX's as containers, using a SimGroup by default will cause much confusion unless you are positive that you only ever want a particular object in a single container.
07/31/2006 (12:31 pm)
Quote:
Well I think the real lesson to take away from this, is that you should use a SimGroup whenever possible. In most cases a SimGroup will make sense for your collection, unless the objects have to be owned by more than one set. Even when you need a SimSet for some reason, it usually makes sense to have a SimGroup somewhere. In that case, if the objects are contained in multiple SimSets, you won't run into issues figuring out at what point you have to delete the objects. You will know that the SimGroup owns the objects, and will be the one place where you will actually delete the objects.
Actually, this will get you in a lot more trouble than you think. In general, you should default to using a SimSet, not a SimGroup unless you expressly need the "delete the group, delete all objects in set" functionality.
The reason is that if you accidentally place an object that is already in one SimGroup into another SimGroup, it will be silently removed from the first SimGroup.
Of course, it depends on what you need to do, but if you are simply using your SimXXX's as containers, using a SimGroup by default will cause much confusion unless you are positive that you only ever want a particular object in a single container.
#15
Also great comments here, very useful.
Edit: Argh, this solved also a problem with my A*. Usually only half of the objects got deleted in my code. Now I know why... :-)
08/10/2006 (11:14 pm)
Woah! What an awesome plan! Thanks a lot for this helpful tips. Although I use TorqueScript for years now, I learned something new here (and this fixed a bug in my game by the way!) :-)Also great comments here, very useful.
Edit: Argh, this solved also a problem with my A*. Usually only half of the objects got deleted in my code. Now I know why... :-)
#16
08/14/2006 (4:43 pm)
Wow, I totally just looked this up so I could use it. Awesome.
#17
function foo()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
%set.clear();
%set.delete();
}
07/17/2007 (6:35 pm)
You can also call the function clear() to remove all of the objects from a simset.function foo()
{
// Create a SimSet
%set = new SimSet();
// Create 10 ScriptObjects and add them to the SimSet
for(%i = 0; %i < 10; %i++)
{
%object = new ScriptObject("object" @ %i);
%set.add(%object);
}
// Print the the names of the object's in the SimSet
for(%i = 0; %i < %set.getCount(); %i++)
{
%object = %set.getObject(%i);
echo("Object in SimSet:" SPC %object.getName());
}
// I'm done, now cleanup
echo("Deleting Objects...");
%set.clear();
%set.delete();
}
#18
05/05/2008 (3:33 pm)
That is incorrect I think. clear() Removes the objects from the simset, do not delete them. The result is like just deleting the simset, ignoring its content. 
#19
I've also been using clear() to delete simsets. Can anyone confirm this deletes everything in the simset? I'll look into this also.
I'm glad I found this post because I believe I have a memory leak in my game from not deleting the script objects in the simset properly.
04/07/2010 (4:47 pm)
If you end a level(i.e scene.endLevel()) and there's a simset with objects from that level, does the simset get deleted? I've also been using clear() to delete simsets. Can anyone confirm this deletes everything in the simset? I'll look into this also.
I'm glad I found this post because I believe I have a memory leak in my game from not deleting the script objects in the simset properly.
Torque 3D Owner Jesse Liles