My Degree Thesis - A I for non-player charachters

#1 By the way you the Torque community are supposed to be my clients for the purposes of my thesis and this AI controller is supposed to be for you all. Your posts will probably be printed out and included in my thesis in some way if they are any good.

Thanks

#2 @CU1

Given your requirements, I believe 4-6 week is enough time for a ROUGH AI that will accomplish what you are proposing to do. I don't think you will have to dab too much in the C++ engine code but you will do most of it in the CS (torque script) code. I know there is plenty of AI info in the forums for owners of the engine so you will probably have to invest in buying the engine if you have not done so yet.

Also, I know the new Advanced 3D Programming All-In-One by Ken Finney covers basic AI so I recommend you buy it. Having said that, if you don't already know Torque, torque script, 3D modeling and rigging, it may take you longer to get to the actual work of your thesis. I suggest you pair up with someone who can take care of working with the engine and implementing you work in the FPS demo, and you concentrate in the actual AI part.

Sounds like a great opportunity to learn AI specifically designed for Torque. It's time Kork learns some new tricks and does something else other than running around stronghold :)

Let us know if there is anything specific that we as a community can help you with and we'll do our best.

#3 Buy the Advanced 3D Programming All-In-One but don't expect it to go into any depth on any particular subject especially not AI. It is a good introduction to the Torque engine if one can stand the tedious tutorials on how to draw bitmaps and what not. A really useful book on AI is Programming Game AI by Example, clearly the best book on the subject I've ever read.
http://www.ai-junkie.com/books/index.html
A tip though, write in C++ instead of script to save on your sanity. Script has some weirdnesses compared to "real" languages but in order to do AI you MUST be able to debug properly. Nothining is as debugging intensive as AI programming and being able to use Visual Studio's debugger and esp. its edit-and-continue feature is a must for any serious work.
Good luck and use the community as much as possible, most here are extremely helpful and well versed in the Torque engine.

#4 4-6 months should be plenty of time to get a good system going. Time is really relative to a few things

- How much you have to learn. How well do you know the Torque Engine? How well do you know Scripting? How well do you know TorqueScript? How well do you know programming? This will definately be a deciding factor in time, learning can be very time intensive so you may find that a majority of your time isn't actually implementing but learning.

- What features are you expecting in your AI system? You seem to cover this fairly well though the biggest question left is pathfinding. This can be one of the biggest challenges in an AI system. Do you plan to use an already existing resource, like the A* one? Do you plan on coding it from scratch (pathfinding is definately something that should be heavily done in the C++ end, at least path generating/calculating... though some interfacing with the scripting language works nicely. Do you plan on doing your own custom solution? Pathfinding is not only a pain to code, but a pain to get working, lots of tweaking and changes after testing... as well as lots of debuging. This is where there can be a misconception that scripting has a failing in this area, in truth (at least from my experience) scripting has a huge strength in tweaking and testing. To test something you don't have to recompile the engine, thats an immense timesaver.


For a college final project my team and I are nearly finished with a Torque game in 2 months (no exageration, in fact probably a week or two less time). This includes AI, pathfinding, flocking, gameplay etc. I am the sole scripter/programmer on it. Fortunately there is a good Torque A* pathfinding resource that I have heavily utilized. Though in between scripting gameplay, exporting assets, and finishing up documentation I have done some very fast AI scripting. I just recently implemented a pure script flocking system that works very well... I can easily add new formations as well. I also designed it heavily around reducing the "thoughts" of each AI unit and put the heavy load on the leader. I did the flocking system in roughly 2-4 days, not solid days though I dabbled in other days as well. Though I am fairly experienced at Torque and TorqueScript. I would strongly suggest you do a lot of it in script, though keeping heavy looping and process intensive methods in the source.

#5 I will be buying Advanced 3D Programming All-In-One by Ken Finney

My knowledge of torque is basic. I dont know Torquescript and will be learning from scratch.

In terms of path planning I want to keep it basic. As I said earlier the plan is to send out loads of pic rays in a fan shape from the NPC. I thought that if a particular pic ray goes through the player then the angle of the pic ray would be known and you would have a rough idea of where the player is. You could then turn the NPC in the direction of the player and get the NPC to run towards the player. IS this ok and is it likely to work?
Also can you find the distance between the player and NPC using the pic ray? If so then I could get the NPC to stop a certain distance from the player so that the NPC doesnt just keep trying to run into the player.


While I wait for my book I am reading up on the Fuzzy Logic part of my system. My thesis supervisor reccomended

The Fuzzy Systems Handbook (Second Edition) by Earl Cox

Its very heavy reading but comes with a cd with some code on it. I will post my fuzzy logic code soon. Ill be coding it seperate from Torque for now.

Thanks for the advice!!

#6 These below tasks are easily done via script, although repeated conditional ray casting via script, would be intensive/slower. C++ would be better

- send out loads of pic rays in a fan shape from the NPC.
- if a particular pic ray goes through the player ... idea of where the player is
- turn the NPC in the direction of the player and get the NPC to run towards the player
- find the distance between the player and NPC using the pic ray
- get the NPC to stop a certain distance from the player so that the NPC doesnt just keep trying to run into the player.

You would need to review the open forums, and get used to what the scripting methods are with the 'free' version, or buy the engine and be able to use/tweak the 'owner' forum resources.

>>Is this ok and is it likely to work?
Yes.

4-6 months should be more than enough for these basic tasks, dependent on your skills & how quickly you pick up on logic/methods/torque basics

#7 I have already got the game engine. My university bought the lisences as part of my degree course. As far as I am aware my university is the first to do so.

#8 UPDATE ON PROGRESS

You might be wondering where Ive been all these days. Well Ive just read most of 3D Game Programming All In One (Ive left a couple of the 3D modelling chapters which I look forward to coming back to when I have more time. I know have a basic knowledge thanks to this great book. Right now Im reading the Ai chapters of Advanced 3DGPAI1 which should get me started in actually doing some coding (At last!!!).

All praise to Kenneth C Finney the provider of gaming wisdom. Its been a long day, I had better get on with reading. Ill keep everyone updated on my progress.

#9 Just saw this

Dont shoot out rays in all directions! Use the container search instead, where you query for Player type objects within a certain radius. Much nicer.

Personally I would do such basic stuff in script mostly. Then later (if you have time), you can port portions back into the engine for performance reasons.

Personally (I have fairly good knowledge of TGE and script) I implemented autonomous steering behaviour for vehicles into the engine in C++ within a weeks time. So it can be done if you know how.

#10 I have scrapped that idea some time ago. The only reason I suggested it is because one of my lecturers who I though knew what he was talking about told me about this idea.

UPDATE ON PROGRESS

WOOHOO Ive got a NPC who chases me, shoots at me and stops before he gets too close. All thanks to Advanced 3D 3DGPAI1. Ive still got to get the NPC to:

run away and then turn around when health is low
As I am doing better than expected jump toward and away from you.

I still havent started my fuzzy logic controller but another one of my lecturers said its easy. In the end I should end up with this:


High health

jump towards
run towards
stand still
run away
jump away

low health


Heres my code so far:

add
exec("./aiNPC.cs");
to
C:\Torque\SDK\example\demo\server\scripts\game.cs

add
aiNPC.cs (See below)
to
C:\Torque\SDK\example\demo\server\scripts\



One question. I used the code from Advanced 3DGPAI1 chapter 7 as a base. That uses the torque demo. I tried inputing the code into the fps demo but I cant even launch the game. Windows just tells me to get lost. This is nothing major but I just want to know why its not working. Thanks!!

#11 @ CU#1, do you have a name ? :-)

I'm also studying Computational Inteligence applied to games for my master degree. I'm very interested in your thesis.

Quote:
1) Use the Torque FPS demo as a base.
2) Give the non-player character (NPC) basic sight by sending out loads of pic-rays from it so it can recognise you.
3) Give the NPC a small brain using a fuzzy logic based system. There will be three stages to this. The NPC can either run toward you, Stand still or run away and this will be triggered by how high the enemy health is.
4) Discuss why unpredictable NPC's in games makes for a better game.

2) What do you mean as sending out loads of pic-rays ? Why do you need to recognise the player.

3) There are two approaches you could use using fuzzy logic, either a fuzzy logic inference system or a fuzzy finite state machine.

Also, your implementation seems a little odd. The point on using fuzzy logic is to obtain a numeric answer for a set of inputs. If your output is discreet : attack, stand and run away, there is no point in using FL. Also, you'll need other input variables. Only the enemy's health is not enough.

Consider the classic sprinkler fuzzy problem. You need to water the garden and you control how much time your sprinkler will be on. You have two input variables : temperature and humidity.
Temperature is : Very low, low, medium, high, very high. Humidity is :dry, mild and humid. Time is : short, normal and long.
Your inference set would be : If temperature is high and humidity is dry then time is long...

Your inference engine would spawn several rules when the temperature is between very low and low ( for instance ) that will create a fuzzy set for the Time variable that will be defuzzified into a number (using the centroid aproximation ).

What I'm trying to say is that there is no fuzzyness in attack, stand and run. Let's say the enemy health triggers a low and medium rule. What would be the half way between attack and stand ?? To attack slowly ?

4) This is one of the basic hipotesis of my thesis. I intend to prove it by giving people two versions of the game, one with traditional AI ( FSMs ) and the other with CI ( fuzzy, neural or genetic algorithms ) and asking people which they find better. But the main problem is defining what "Better" is. Have you thought about this ?

#12 Hi C.U. 1.

Did you mean to post the game.cs file? from you post, it seems that you wanted to post your ainpc.cs file.

#13 Better is a closer approximation to human play. Random-seeming behavior that may not be random.

#14 To

Bruno Grieco (Member)

My name is Rehan Saeed. Its nice to see someone care.

2) As I have stated above just 2 weeks ago I was new to Torque and when I asked one of my lecturers who I thought knew what he was talking about, he suggested this idea. I have since scrapped this idea. Although at a very basic level from what I understand the code I have used above (Most of it from Advanced 3DGPAI1) uses this technique to detect when to shoot at the player.

3) I must admit I havent read up on the fuzzy logic as much as I should have thus far, being so buisy with Torque. But this is the plan so far:

If you look at the code above the NPC's health is not the only factor. You have:

aggression
alertness
attention
range
$MAX_CHASER_SPEED = 0.6;
%tgtEnergyLevel //target energy level
%resLength = vectorLen(%resultant); //AI's distance from player


I understand what you mean by

"What would be the half way between attack and stand ?? To attack slowly ?"

Well, why not. We have a speed attribute to the NPC. Once I get the NPC jumping around I'll have to deal with how often the NPC jumps too.

4) Good question. My thesis is based on the idea that more unpredictable AI provides a better experience for the player (Thats PART of the reason why people love playing games like Battlefield 2. You just have no idea what the player is going to do next and you cant learn a set routine when attacking another player). You have to constantly adapt and try new techniques to find the best way to deal with the enemy.

The way I am going to test my NPC is by giving it to the gamers. Getting them to tell me what they think. Its very difficult to measure the unpredictablility of a NPC or indeed to measure the level of game play.

All I can think about is getting the player to rate various things out of 10 or get the player to play the game and pausing the game every now and then to get the player to predict what is going to happen next. Hope that helps!

Have a look at my code, its pretty basic. If you havent already done so I would advise you to get hold of

AI game programming by example (I am told it gives a very good idea of fuzzy logic. I have yet to get holdof this.)
and Advanced 3D game programming all in one from which I used some code to get the above.

It is nice to see someone else doing something similar. I hope I have answered your questions :)

#15 Hello Rehan,

One tip : when posting code you can use the "["code"]" and "["\code"]" ( remove the quotes ) identifiers. using them will maintain the indentation making the code easier to read. You may also edit the past posts to include them.

Casting all those rays work, but check out how to create a container. It's not that hard. You create them with a radius and they will contain all objects that are in that radius. You'll have better performance that way.

Once you get going with your basic FSM for the NPCs, you should dive into the fuzzy logic book. I have Finney's book but it doesn't really talk of Fuzzy Logic. I haven't seen AI game programming by example. But as a rule of thumb, "commercial" books don't focus much on this kind of approach. They use a more classic FSM approach.

A state isn't something fuzzy. It's the opposite, it's crisp. You are either in a state "attacking" or in a state "standing". You should choose an output variable that is fuzzy. Something like speed, you are running faster than normal, but it still isn't fast. It's something in between.

Also, be carefull if you use too many variables to determine what your NPC should do. This will result on an explosion of inference rules.

#16 Thanks Bruno for the advice. Im not too bothered with performance at the moment but Ill look into it later. After I read your previous post I did dive into the fuzzy logic book I told you about. Its pretty good (40 pages) including a working code example.

Speed is something I wanted to fuzzify, so to speak.

Rehan

#17 Ive been trying to get the ai NPC to jump around but as yet have been unable to overcome a problem.

when I insert the code

%obj.pushTask("playThread(0,\"dance\")");

he chases me while dancing around like a wild man. The dance anim is continuous and so keeps repeating. When I try a wave:

%obj.pushTask("playThread(0,\"celwave\")");

the NPC waves once and then chases me withought moving its legs. When I try

%obj.pushTask("playThread(0,\"jump\")");

I see nothing. This is probably because I dont notice him jumping.

Any ideas?

Heres the latest code. Ive got the unblock working, the NPC now runs off in random directions when he gets stuck.

#18

$debugSwitch[onStuck] = false;
$debugSwitch[unBlock] = false;
$debugSwitch[movePlayer] = false;
$debugSwitch[checkForThreat] = false;
$debugSwitch[DoScan] = false;
$debugSwitch[openFire] = false;
$debugSwitch[ceaseFire] = false;
$debugSwitch[onTargetEnterLOS] = false;
$debugSwitch[onTargetExitLOS] = false;
$debugSwitch[spawn] = false;
$debugSwitch[Equip] = false;
$debugSwitch[GetTargetRange] = false;
$debugSwitch[getClosestEnemy] = false;
$debugSwitch[CreateBots] = false;
$debugSwitch[pickAISpawn] = false;
$debugSwitch[GetBearing] = false;
$debugSwitch[GetHeading] = false;
$debugSwitch[GetRelativeBearing] = false;
$debugSwitch[CheckArcOfSight] = false;
$debugSwitch[Misc] = true;

$ARC_OF_SIGHT = 250;

$MIN_SCAN_GAP = 1000;
$MAX_SCAN_GAP = 20000;
$MIN_TRIGGER_HOLD = 100;
$MAX_TRIGGER_HOLD = 200;
$MIN_ITCHY_FINGER = 2000;
$MAX_ITCHY_FINGER = 10000;
$MAX_THREAT_ENGAGE_RANGE = 100;
$MAX_AGGRESSIVENESS = 100;
$MAX_ATTENTION_LEVEL = 200;
$MAX_ALERTNESS = 100;
$STATIONARY = 0.0;
$MAX_CHASER_SPEED = 0.6;
$PLAYER_TYPE = "Guard";
$MIN_DISTANCE_TO_PLAYER = 10;
$MAX_FLEEING = 2000;

$cardinalDirection[0] = "0 10000 0";    // N
$cardinalDirection[1] = "6000 6000 0";  // NE
$cardinalDirection[2] = "10000 0 0";    // E
$cardinalDirection[3] = "6000 -6000 0"; // SE
$cardinalDirection[4] = "0 -10000 0";   // S
$cardinalDirection[5] = "-6000 -6000 0";// SW
$cardinalDirection[6] = "-6000 0 0";    // W
$cardinalDirection[7] = "-6000 6000 0"; // NW
$NUM_CARDINALS = 8;


datablock PlayerData(AIPlayerDB : LightMaleHumanArmor)
{
   className = "Armor";
   maxInv[CrossbowAmmo] = 500000;  
};

function AIPlayerDB::checkForThreat(%this,%obj)
{
  DebugPrint( "%this:"@%this@"~AIPlayerDB::checkForThreat  (from:"@%obj@")", "checkForThreat");
  if(!isObject(%obj))
    return;
  
  if (isObject( %obj) )
  {
    %idx = %obj.getClosestEnemy();
    if (%idx < 0)
      return 0;
    %target = ClientGroup.getObject( %idx ); /// player objects as targets
    if ( !%obj.CheckArcOfSight(%target.player) )
      return;
    if (%obj.attentionLevel>0)  // if attentionLevel is non-zero, keep looking at max range
    {
       %testRange = %obj.range * (%obj.aggression*2);
    }
    else
    {
       %testRange = %obj.range;
    }
  }
  else
  {
    return 0;
  }
  
  if (%target.player == %obj.currentTarget)
  {
    if (isObject( %obj) )
    {
       if ( %obj.GetTargetRange(%target.player) <  %testRange)
       {
          return %target.player;
       }
    }
    else
       return 0;
  }
  else     /// new threat
  {
    if (isObject( %obj) )
    {
       if ( %obj.GetTargetRange(%target.player) <  %testRange)
       {
          return %target.player;
       }
    }
    else
       return 0;
  }
  DebugPrint( "no threat (from:"@%obj@")", "checkForThreat");
  return 0;
}

#19

function AIPlayerDB::DoScan(%this,%obj)
{
  DebugPrint("%this:"@%this@"~AIPlayerDB::DoScan (from:"@%obj@")", "DoScan");
  if(!isObject(%obj))
    return;
  cancel(%this.scheduledCheck);
  if (%obj.attentionLevel<=0)  // if attentionLevel is non-zero, keep looking in same direction
    %obj.attentionLevel=0;
  else
    %obj.attentionLevel--;
  if (%obj.attentionLevel==0)  // if attentionLevel is non-zero, keep looking in same direction
  {
   %look = getRandom($NUM_CARDINALS-1);
   if (%this.look != %look)
      %this.look = %look;
   else
   {
      %this.look = %look + 1;
      if (%this.look < $NUM_CARDINALS-1)
         %this.look++;
      else
         %this.look = 0;
   }
  }
  %obj.setAimLocation($cardinalDirection[%this.look]);
  if ( (%tgtPlayer = %this.checkForThreat(%obj)) != 0)
  {
    if (%obj.currentTarget)
    {
       if (%obj.currentTarget==%tgtPlayer)
       {
          DebugPrint( "STILL A THREAT (from:"@%obj@")", "DoScan");
          %obj.setAimObject( %tgtPlayer );
					%this.movePlayer(%obj, %tgtPlayer);					
          %obj.attentionLevel = %obj.attention;
       }
       else
       {
          DebugPrint( "CHANGED THREAT (from:"@%obj@")", "DoScan");
          %obj.currentTarget =  %tgtPlayer;
          %obj.setAimObject( %obj.currentTarget );
					%this.movePlayer(%obj, %tgtPlayer);
       }
     }
     else
     {
       DebugPrint( "NEW THREAT!! (from:"@%obj@")", "DoScan");
       %obj.setAimObject( %tgtPlayer );
			 %this.movePlayer(%obj, %tgtPlayer);  
       %obj.currentTarget =  %tgtPlayer;
       %obj.attentionLevel = %obj.attention;
     }
     //%obj.pushTask("playThread(0,\"jump\")");
     %this.schedule($MIN_SCAN_GAP+getRandom($MAX_SCAN_GAP/%this.alertness), "playThread(0,\"celwave\")", %obj);
 	}
  else
  {
    if (%obj.getAimObject)
    {
       %obj.clearAim();
            DebugPrint( "> %obj.clearAim (from:"@%obj@")", "DoScan");
       %obj.currentTarget = 0; // forget this target
    }
    %this.nextScan = %this.schedule($MIN_SCAN_GAP+getRandom($MAX_SCAN_GAP/%this.alertness), "doScan", %obj);
  }
}

#20

function AIPlayerDB::movePlayer(%this, %obj, %tgtPlayer)
{
	
	%tgtPosition = %tgtPlayer.getPosition();		//player position
	%myPosition = %obj.getPosition();						//NPC position
	
	
	%resultant = vectorSub(%tgtPosition, %myPosition);
	%resLength = vectorLen(%resultant);									//AI's distance from player
	
	%tgtEnergyLevel = %tgtPlayer.getDamagePercent();					//damage percentage of player out of one
	%myEnergyLevel = %obj.getDamagePercent();								//damage percentage of ai out of one
	
	%mode = 0; //getRandom(2);																																//Decide what to do
	if (%mode == 0)
	{
		if (%resLength >= $MIN_DISTANCE_TO_PLAYER)																					//ATTACK PLAYER
		{
			%obj.setMoveSpeed($MAX_CHASER_SPEED);
  		%obj.setMoveDestination(%tgtPlayer.getPosition());
  		%obj.nextBlockCheck = %this.schedule($MAX_SCAN_GAP*2, "unBlock", %obj, %myPosition, $MAX_FLEEING, 1);
		}
	}
	else if (%mode == 1)																																	//STAND STILL AND ATTACK
	{
	}
	else if (%mode == 2)																																	//RUN AWAY
	{
		%this.setRandomDestination(%obj, %myPosition, $MAX_FLEEING, 2);
	}
}

function AIPlayerDB::unBlock(%this, %obj, %myPosition, %distance, %cardinalRange)
{
	if (!isObject(%obj))
		return;
	cancel(%obj.nextBlockCheck);
	%this.setRandomDestination(%obj, %myPosition, %distance, %cardinalRange);  
	error("UNBLOCKING");
}

function AIPlayerDB::onReachDestination(%this, %obj)
{
	cancel(%obj.nextBlockCheck);
	%this.nextScan = %this.schedule($MIN_SCAN_GAP+getRandom($MAX_SCAN_GAP/%this.alertness), "doScan", %obj);
}

function AIPlayerDB::setRandomDestination(%this, %obj, %myPosition, %distance, %cardinalRange)			//get the NPC to run in all random directions
{																																																		//or away from the player is he sees him.
	%randomDir = getRandom($NUM_CARDINALS - 1);
	%look = %this.look;
	while (%randomDir > (%look + %cardinalRange) && %randomDir <(%look - %cardinalRange))
	{
		%randomDir = getRandom($NUM_CARDINALS - 1);
	}
	%cardDirection = $cardinalDirection[%randomDir];						//dont run back in the same direction you are looking.

	for (%index = 0; %index <= 2; %index++)								//for each axis of the NPC's direction
	{
		%axis = GetWords(%cardDirection, %index, %index);  //select one axis value
		if (%axis > 0)
		{
			%axis = %axis/distance;

			if (index == 0) 
			{
				%cardDirection = %axis SPC GetWords(%cardDirection, 1); 
			}
			else if (index == 1) 
			{
				%cardDirection = GetWords(%cardDirection, 0, 0) SPC %axis SPC GetWords(%cardDirection, 2); 
			}
			else if (index == 2) 
			{
				%cardDirection = GetWords(%cardDirection, 0, 1) SPC %axis; 
			}
		}
	}
	%destination = vectorAdd(%cardDirection, %myPosition);
	%obj.setMoveSpeed($MAX_CHASER_SPEED);
	%obj.setMoveDestination(%destination);
}