[Fixed]Bot Kills Player Fatal Crash.
by CSMP · in Torque Game Engine · 09/02/2009 (1:32 am) · 18 replies
Environment:
Improved AI Guard Resource
TGE1.5.2
Recently while playing in a level with about 20 AI I had a problem, After the AI killed me my game completely crashed: No MS Error, No TGE Error, No real console info, do not pass go and go directly to desktop... And to the desktop I went! :(
Well it happened a crapload more times before I figured out that the AI killing me was the actual culprit... Thank God. (I just installed other resources, please dont hurt me feature creep!!!)
Well after testing multiple more times and with the help of "trace(1);"(thank you for those that referance this in posts as I always forget about the basics!), I narrowed it down to this piece of code:
Entering AIPlayer::GetClosestHumanInSightandRange(2198, 2198)
The crash ALWAYS happens "Entering" the GetClosestHumanInSightandRange function and it almost looks like (with trace) that only the 2198 AI is getting all the "AI::Think" attention prior to the crash.(Possibly Causing!?)
And it happened while respawing or staying in the death camera too long, though it seems the more AI the easier the crash.
Hopefully this will help someone thats getting this "Unkown Error"(as it was before I found out the AI was doing this)
I Don't mind this problem much as there are multiple other AI resources and this is an area of code that I don't mind trying to debug myself, however I hope if anyone comes across this problem it helps. :)
Improved AI Guard Resource
TGE1.5.2
Recently while playing in a level with about 20 AI I had a problem, After the AI killed me my game completely crashed: No MS Error, No TGE Error, No real console info, do not pass go and go directly to desktop... And to the desktop I went! :(
Well it happened a crapload more times before I figured out that the AI killing me was the actual culprit... Thank God. (I just installed other resources, please dont hurt me feature creep!!!)
Well after testing multiple more times and with the help of "trace(1);"(thank you for those that referance this in posts as I always forget about the basics!), I narrowed it down to this piece of code:
Entering AIPlayer::GetClosestHumanInSightandRange(2198, 2198)
The crash ALWAYS happens "Entering" the GetClosestHumanInSightandRange function and it almost looks like (with trace) that only the 2198 AI is getting all the "AI::Think" attention prior to the crash.(Possibly Causing!?)
And it happened while respawing or staying in the death camera too long, though it seems the more AI the easier the crash.
Hopefully this will help someone thats getting this "Unkown Error"(as it was before I found out the AI was doing this)
I Don't mind this problem much as there are multiple other AI resources and this is an area of code that I don't mind trying to debug myself, however I hope if anyone comes across this problem it helps. :)
#2
09/02/2009 (5:28 pm)
ATM, I've disabled the AI and was thinking about diffing this and other AI resources for a solution, specifically the mentioned AI State. 
#3
*edit*
ok i see. it seems as if its searching for itself since it is called as
%this.GetClosestHumanInSightandRange(%obj)
and if it is dead then this may cause the function crash.. i guess :)
09/02/2009 (8:54 pm)
looks like it sees the object and the client as the same thing?? im shooting in the dark here trying to remember how that function worked*edit*
ok i see. it seems as if its searching for itself since it is called as
%this.GetClosestHumanInSightandRange(%obj)
and if it is dead then this may cause the function crash.. i guess :)
#4
and this is the last AI::Think call until it crashes and repeats so many times that it creates strange artifacts in the console report and scrolls so far to the right side in the console report that it loses the ability to correctly print the functions:
Edit: both of these codeblocks are of the AI::Think function after the player is killed.
09/02/2009 (9:49 pm)
strange, because this "seems" to be a working AI::Think call:Entering AIPlayer::Think(2397, 2397) Entering ShapeBase::getInventory(2397, mp5Ammo) Leaving ShapeBase::getInventory() - return 4 Entering AIPlayer::GetClosestHumanInSightandRange(2397, 2397) Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1 Leaving AIPlayer::Think() - return 2397
and this is the last AI::Think call until it crashes and repeats so many times that it creates strange artifacts in the console report and scrolls so far to the right side in the console report that it loses the ability to correctly print the functions:
Entering AIPlayer::Think(2405, 2405)
Entering ShapeBase::getInventory(2405, mp5Ammo)
Leaving ShapeBase::getInventory() - return 29
Entering AIPlayer::GetClosestHumanInSightandRange(2405, 2405)
Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1
Entering AIPlayer::sidestep(2405, 2405)
Entering AIPlayer::GetClosestHumanInSightandRange(2405, 2405)
Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1
Entering AIPlayer::sidestep(2405, 2405)
Entering AIPlayer::GetClosestHumanInSightandRange(2405, 2405)
Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1
Entering AIPlayer::sidestep(2405, 2405)
Entering AIPlayer::GetClosestHumanInSightandRange(2405, 2405)
Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1
Entering AIPlayer::sidestep(2405, 2405)
Entering AIPlayer::GetClosestHumanInSightandRange(2405, 2405)
Leaving AIPlayer::GetClosestHumanInSightandRange() - return -1
Entering AIPlayer::sidestep(2405, 2405)
...
...//and so on and so on until UltraEdit displays the following...
...
o66-±ªA
ë96-±ªA
“96-±ªA
ë96-±ªAEdit: both of these codeblocks are of the AI::Think function after the player is killed.
#5
09/02/2009 (11:05 pm)
This looks really like there's a loop going *ahem* loopy in there... I might take a look at the resource to see how it's using that function. See, it's returning -1 each time... that seems to suggest it's not finding any humans in sight and range. If not, then it should stop, not try again. 
#6
09/02/2009 (11:09 pm)
Not sure if related but I remember having this crash-on-player-death due to a custom death animation. Can't remember if it was not all the bones being animated or whether it was down to a lack of ground frames.
#7
And notice that in the working AI::Think call theres no call to sidestep, strange...
Though the crash always happens on the:
Edit: I've seen the posts on the DeathAnimation bug when I was searching for this problem, This bug only happens when the AI kills the player and the Trace() results I have gotten are pointing to the AI as well, thanks for the help though!
09/02/2009 (11:10 pm)
Ya, Its also calling ::sidestep which should only happen when shot!?And notice that in the working AI::Think call theres no call to sidestep, strange...
Though the crash always happens on the:
Entering AIPlayer::GetClosestHumanInSightandRange
Edit: I've seen the posts on the DeathAnimation bug when I was searching for this problem, This bug only happens when the AI kills the player and the Trace() results I have gotten are pointing to the AI as well, thanks for the help though!
#8
09/02/2009 (11:10 pm)
Here is that function in the regular AiGuard resource. I believe the one you are using is supposed to be better? I don't know. You can use this to check your version. I know that with the regular AiGuard resource, I have 40 bots running around and haven't had a problem.//The next few lines again have the bot check for a target and attack if need be.
%tgtid = %this.GetClosestHumanInSightandRange(%obj);
if(%tgtid >=0)
{
%obj.action = "Attacking";
%this.ailoop=%this.schedule(100,"Think" , %obj);
}
else
#9
Also, Scooby was right in saying that the AI script seems to target clients, and then use their player to target and such.
Wait, what was I saying... this can't be a looping problem, can it? Because all the calls stack up in increasing depth - if this were an infinite loop they'd be at the same call depth. What I'd guess now (:P) is that some method is recursively calling methods that mean you get stuck caling the same methods over and over again recursively.
EDIT: First clue. AIPlayer::sidestep does call getClosestHumanInSightAndRange.
Found your problem, I think.
EDIT: Hmm, however %newLoc should be different every time. Unless there's a specific spot where the AI is surrounded by stuff, or for some reason the random generation isn't working right.
09/02/2009 (11:33 pm)
Odd, I don't seem to see any loops in AIPlayer::Think at all. Sidestep and getClosestHmanInSightAndRange are both called in cases "Returning" and "RetrievingItem". I'm guessing your bots would be in "Returning" after they've killed you.Also, Scooby was right in saying that the AI script seems to target clients, and then use their player to target and such.
Wait, what was I saying... this can't be a looping problem, can it? Because all the calls stack up in increasing depth - if this were an infinite loop they'd be at the same call depth. What I'd guess now (:P) is that some method is recursively calling methods that mean you get stuck caling the same methods over and over again recursively.
EDIT: First clue. AIPlayer::sidestep does call getClosestHumanInSightAndRange.
Found your problem, I think.
//There is no target
else
{
//Line of sight test for the position the bot wants to sidstep to
%eyeTrans = %obj.getEyeTransform();
%eyeEnd = %newLoc;
%searchResult = containerRayCast(%eyeTrans, %eyeEnd, $TypeMasks::PlayerObjectType | $TypeMasks::StaticTSObjectType |
$TypeMasks::TerrainObjectType | $TypeMasks::ItemObjectType | $TypeMasks::InteriorObjectType | $TypeMasks::StaticObjectType, %obj);
%foundObject = getword(%searchResult,0);
if (%foundObject > 0)
{
%this.sidestep(%obj);
return;
}
//Set the bot to look in the direction that it is moving.
else
%obj.setaimlocation(%newloc);
}This is called if we can't find a client in sight and range. Notice it calls sidestep on itself? I reckon that it just keeps calling sidestep on itself recursively, never finding a client, but always finding an object with its raycast. Indeed, if it finds an object *once* with its raycast, it will enter the infinite loop.EDIT: Hmm, however %newLoc should be different every time. Unless there's a specific spot where the AI is surrounded by stuff, or for some reason the random generation isn't working right.
#10
Also theres this code in the getClosestHmanInSightAndRange that is called:
so it seems as if my client is no longer a "valid" client and somehow sending it into a confused rampage of console spam.
Edit: because of the fact that the more AI around the quicker it crashes I think you are correct about it finding an object and going "loopy". (pardon the pun, lol)
???
09/02/2009 (11:39 pm)
Ok, the Returning method sounds appropriate for the situation..Also theres this code in the getClosestHmanInSightAndRange that is called:
//The for-next loop cycles through all of the valid clients
for(%i=0; %i < %count; %i++)
{
%client = ClientGroup.getobject(%i); //Get the client info for the client at index %i
//If the client is invalid then the function bails out returning a -1 value, for no target found.
if (%client.player !$= "" && %client.player > 0)
{
....
....so it seems as if my client is no longer a "valid" client and somehow sending it into a confused rampage of console spam.
Edit: because of the fact that the more AI around the quicker it crashes I think you are correct about it finding an object and going "loopy". (pardon the pun, lol)
???
#11
09/02/2009 (11:50 pm)
What I'd do as a quick test is comment out the call to sidestep in the script I posted above. That'll break the AI functionality slightly, but if you don't get crashes any more you'll know you found the culprit.
#12
09/02/2009 (11:53 pm)
10-4, I'll check it out now.
#13
Tried it 4 times with my basic.exe and 4 times with my advanced.exe. (I'm using the dual project setup you mentioned for the TMK)
Looks like you found the problem code, thanks for going way beyond to help me out!
Edit: Tested some more, and still no bug... awesome!
09/03/2009 (12:27 am)
Wow Daniel, oh mighty debugger, I just tried it 8-times and no bug...Tried it 4 times with my basic.exe and 4 times with my advanced.exe. (I'm using the dual project setup you mentioned for the TMK)
Looks like you found the problem code, thanks for going way beyond to help me out!
Edit: Tested some more, and still no bug... awesome!
#14
09/03/2009 (1:39 am)
Glad it worked for you! Now all that's left is to find out why it doesn't work and fix it ;)
#15
Either way, for the moment I'm glad its not crashing the engine, and its narrowed down to a relatively small codeblock!
Thanks again Daniel!
@Daniel: If you dont mind me asking... I've been wondering what your progress was on the WarEngine?
09/03/2009 (1:48 am)
Lol, ya its too easy to kill the AI now that they are not sidestepping.Either way, for the moment I'm glad its not crashing the engine, and its narrowed down to a relatively small codeblock!
Thanks again Daniel!
@Daniel: If you dont mind me asking... I've been wondering what your progress was on the WarEngine?
#16
I've done a ton of design work, though, and I don't think many of the other features will take as much work as the Player. Hopefully I'm doing the hardest thing first.
And then, of course, my time is split between that and another project... code changes for which will probably make it into the WarEngine in any case.
Anyway, that was a bit of an off-topic personal meander :P. Back on topic: my pleasure. ;)
(I think I'll get around to bloging some design goals for the WarEngine soon.)
09/03/2009 (3:24 am)
Mm, gradual and small :P. I keep beating my head against my custom Player class... I've got some network issues after merging Player and AIPlayer into one class, and aside from that I keep seesawing on my plans for collision detection and movement. I think I've made a decision - but I still need to implement it ;P.I've done a ton of design work, though, and I don't think many of the other features will take as much work as the Player. Hopefully I'm doing the hardest thing first.
And then, of course, my time is split between that and another project... code changes for which will probably make it into the WarEngine in any case.
Anyway, that was a bit of an off-topic personal meander :P. Back on topic: my pleasure. ;)
(I think I'll get around to bloging some design goals for the WarEngine soon.)
#17
Thanks again, I replaced the problem code with the stock AIGuard code and it "seems" to be working(now watch it'll bite me in the a** tomorrow...lol), I may try and find out what was creating the neverending loop, but for the most part, it works and im happy with it!
Thanks Mike, Steve and Scooby as well... I really didnt expect so much help, I just wanted others to know what might have been causing the fatal crash that I was experiencing...
Good Ol' GG Community at its best!
09/03/2009 (3:40 am)
Considering, you helped solve the topic, I really dont mind hearing what your up to... can't wait for your blog!Thanks again, I replaced the problem code with the stock AIGuard code and it "seems" to be working(now watch it'll bite me in the a** tomorrow...lol), I may try and find out what was creating the neverending loop, but for the most part, it works and im happy with it!
Thanks Mike, Steve and Scooby as well... I really didnt expect so much help, I just wanted others to know what might have been causing the fatal crash that I was experiencing...
Good Ol' GG Community at its best!
#18
This issue and many others have already been corrected in The Universal AI Starter Kit, which should be up on the dev store relatively soon. I don’t have any plans to port this fix back into the Improved AI Guard Unit resource, but there are one or two other fixes that I will be porting back.
09/03/2009 (7:20 pm)
I know I'm a little late, but I thought I'd chime in anyway. An infinite loop in the sidestep function is a rare, but known issue. It's usually caused when the bot gets stuck in a small area.This issue and many others have already been corrected in The Universal AI Starter Kit, which should be up on the dev store relatively soon. I don’t have any plans to port this fix back into the Improved AI Guard Unit resource, but there are one or two other fixes that I will be porting back.
Torque Owner Daniel Buckmaster
T3D Steering Committee