NeuralNet FSM mapping
by Craig Fortune · 03/02/2007 (3:34 pm) · 3 comments
First off, wow - I'm blogging like properly regularly :D I for once kept my promise of keeping my blogs upto date! Secondly...
Neural Nets
With my Neural Net system one of the key design aspects was that I was interested in using the NN to initiate complex behaviours, rather then using the NN to do the movement etc itself. What this means is that I'll be using the outputs of my NN to decide upon what state to move to within a FSM(Finite State Machine) type setup.
What you need for this is a way to map the outputs to the states. For this end I've created a class called "StateHolder", its a simple class that pretty much holds structs that contain data about a state (notably: its name), and the states it is linked to within my FSM. It is a very simple setup in essence but the thing that makes this really nice is that you can register the states and the linked states from script very easily. Each state (in the order you register them) is then mapped to the NN's output. For instance the first state you map maybe called "flee" and that will be mapped to output[0], attack mapped to output[1] and so on.
The key to this system is that whenever you process the NN, (whenever your AI Agent "thinks") you order the outputs it produces in decending numerical order. You then attempt to move to the highest valued output (remember: it is mapped to a FSM state) thats available. If the FSM doesn't have this link defined from your current state (we have this data readily available due to our StateHolder class), you try the next highest ranked output and so on until you change state or decide to do no state changing.
Sounds a bit weird and stuff but simply put all I'm doing is changing to the output (state) that the NN most desires to be in. (Remember: a NN gets trained towards processing the inputs of a situation into outputs that will give it the highest gain - if fleeing is a high gain output, then output[0] (if thats what you have fleeing behaviour mapped to) should be trained up to give a high output when a situation you should want to flee occurs... make sense? Kind of? No? Read it again :D)
Instead of normal NN thinking of:
think of this:
So now this system is in place I can simply code the different "behaviours" into the states within script and start training some NNs. *grin* Of course theres a few more complexities I have included in this, like having a cooldown on state changing (I dont want to be flicking inbetween states lots - so a cooldown on state changing should fix this and smooth it out)
Possible Torsion Bug?
As a complete off topic and probably not worthy of sticking in a .plan... has anyone had any weird problems with Torsion ignoring breakpoints in your script? I've had it decide to not bother with breakpoints in specific files but be perfectly ok with breaking on some breakpoints in other files... weird Huh? Just wondering if I'm alone here and its my system being weird or whether there is a bug is Torsion? I'd rather not file a bug report until I'm somewhat sure its not me to blame.
Neural Nets
With my Neural Net system one of the key design aspects was that I was interested in using the NN to initiate complex behaviours, rather then using the NN to do the movement etc itself. What this means is that I'll be using the outputs of my NN to decide upon what state to move to within a FSM(Finite State Machine) type setup.
What you need for this is a way to map the outputs to the states. For this end I've created a class called "StateHolder", its a simple class that pretty much holds structs that contain data about a state (notably: its name), and the states it is linked to within my FSM. It is a very simple setup in essence but the thing that makes this really nice is that you can register the states and the linked states from script very easily. Each state (in the order you register them) is then mapped to the NN's output. For instance the first state you map maybe called "flee" and that will be mapped to output[0], attack mapped to output[1] and so on.
The key to this system is that whenever you process the NN, (whenever your AI Agent "thinks") you order the outputs it produces in decending numerical order. You then attempt to move to the highest valued output (remember: it is mapped to a FSM state) thats available. If the FSM doesn't have this link defined from your current state (we have this data readily available due to our StateHolder class), you try the next highest ranked output and so on until you change state or decide to do no state changing.
Sounds a bit weird and stuff but simply put all I'm doing is changing to the output (state) that the NN most desires to be in. (Remember: a NN gets trained towards processing the inputs of a situation into outputs that will give it the highest gain - if fleeing is a high gain output, then output[0] (if thats what you have fleeing behaviour mapped to) should be trained up to give a high output when a situation you should want to flee occurs... make sense? Kind of? No? Read it again :D)
Instead of normal NN thinking of:
Inputs -> Processing -> Outputs
think of this:
Inputs -> Processing -> State Choice
So now this system is in place I can simply code the different "behaviours" into the states within script and start training some NNs. *grin* Of course theres a few more complexities I have included in this, like having a cooldown on state changing (I dont want to be flicking inbetween states lots - so a cooldown on state changing should fix this and smooth it out)
Possible Torsion Bug?
As a complete off topic and probably not worthy of sticking in a .plan... has anyone had any weird problems with Torsion ignoring breakpoints in your script? I've had it decide to not bother with breakpoints in specific files but be perfectly ok with breaking on some breakpoints in other files... weird Huh? Just wondering if I'm alone here and its my system being weird or whether there is a bug is Torsion? I'd rather not file a bug report until I'm somewhat sure its not me to blame.
About the author
#2
Scenario 1: AI Agent is in attack state and attacking another player but losing the fight, fleeing would be the higher gain.
Scenario 2: AI Agent is in attack state and attacking another player and winning the fight, continue attacking would be the higher gain.
03/02/2007 (11:51 pm)
Sounds cool Craig, I think Ive got the grasp of it, correct me if Im wrong.Scenario 1: AI Agent is in attack state and attacking another player but losing the fight, fleeing would be the higher gain.
Scenario 2: AI Agent is in attack state and attacking another player and winning the fight, continue attacking would be the higher gain.
#3
The FSM stuff is a very small piece of tech and highly specific to my NN implementation - so no its not worth selling it - or for that matter trying to convert it into something "sellable"
@Andy:
Exactly. The NN implicitly assesses what is the highest gain by the training you provide it. A highly trained NN should be able to assess a situation better and ultimately make better decisions.
03/02/2007 (11:55 pm)
@Eric:The FSM stuff is a very small piece of tech and highly specific to my NN implementation - so no its not worth selling it - or for that matter trying to convert it into something "sellable"
@Andy:
Exactly. The NN implicitly assesses what is the highest gain by the training you provide it. A highly trained NN should be able to assess a situation better and ultimately make better decisions.
Employee Eric Preisz
Can't wait to see it. Are you interested in licensing it when you are done? I am building a scenario generator for creating dynamic scenes. I would rather buy something that works than finish what I'm building.