Game Development Community

RC3: Big precision error when saving levels

by Tom Spilman · in Torque Game Builder · 06/19/2006 (4:32 pm) · 5 replies

There is a precision bug in RC3 when saving levels. Currently in mathTypes.cc you have this function for converting a Point2F to a string:

ConsoleGetType( TypePoint2F )
{
   Point2F *pt = (Point2F *) dptr;
   char* returnBuffer = Con::getReturnBuffer(256);
   dSprintf(returnBuffer, 256, "%.3g %.3g", pt->x, pt->y);
   return returnBuffer;
}

This doesn't work well with fractional values. For instance when sprite was placed at 166.3, 94.05 it converts this to 166, 94.1 when saving the level. So it would look right in LevelBuilder, but when we went to play the level the objects would have moved. This totally screws you when your trying to line up objects with any level of precision.

The problem is that for %g the precision is not decimal digits, but significant digits. The fix is to use %.3f which properly writes out 166.3, 94.05 in all my test cases.

Also note this bug effects ConsoleGetType( TypePoint2FVector ) in t2dUtility.cc which needs to be changed to %.3f from %.3g.

About the author

Tom is a programmer and co-owner of Sickhead Games, LLC.


#1
06/19/2006 (5:12 pm)
Crud, you mean that g uses signifcant digits instead of decimal places?!

Added to known issues.

Cheers,
-Justin
#2
06/20/2006 (8:40 am)
Thank goodness, this has been bugging me for quite some time. I just figured it was supposed to be sig figs!
#3
06/20/2006 (10:33 am)
I had hell trying to line everything up in the level editor too, so I ended up just calculating where they were suppose to be and seting everything up in script . . . I'm not to impressed with the level editor
#4
06/20/2006 (10:36 am)
Well... bugs happen. It sucks that this one took so long to get reported and fixed. Anyway we love the LevelBuilder... watch for our plan post this evening for details. ;)
#5
07/14/2006 (1:54 pm)
@James, well I guess you not being too impressed with the level editor is ok since it would imply that you were atleast 'impressed' just not too :)

Tom,

This has been fixed and will be in the 1.1.1 release, thanks again!

Cheers,
-Justin