by date
Complete AI System for Torque
Complete AI System for Torque
| Name: | Dan Keller | |
|---|---|---|
| Date Posted: | Mar 10, 2008 | |
| Rating: |  4.0 out of 5 | |
| Public: | YES | |
| Comments: | YES | |
| RSS Feed: |  or Subscribe with  . | |
| Profile Page: | View profile page for Dan Keller |
Blog post
Well, it's done. I've finally finished my ai system, except for a few minor things. (Well, I actually finished it a while ago, but I've been too busy to post it up here.) My basic goals in this project have been to make an opponent ai that is flexible, extensible, and, once completed, easy to implement. This has probably been one of my most ambitious coding projects yet, and I'm glad to say, I'm quite satisfied with the results. It uses a three-layered system: At the top layer, a genetic algorithm causes the bot population as a whole to learn and adapt over time. In the middle layer, a each bot uses a neural net to evaluate relevant input information and make a decision as to what to do next. And at the bottom layer, an A* pathfinder tells the bots how to get where they want to go.
Now down to the nitty-gritty description. When the game starts, the AIManager is created. It loads the neural net gene pool from a file, or creates a blank pool. (It has to be randomized from the console because of certain annoyingnesses with global object constructors.) When an AIPlayer "bot" is created, the AIManager combines two random nets from the better half of the gene pool, and gives it to the bot.
Then, the thinking begins. Each bot's thinking function is called, every other tick, by the AIManager. (I'm considering, for efficiency's sake, doing this some other way.) The bot converts all the information is uses into values from -1 to 1. There are currently 7 inputs: Whether the bot is doing anything, how long it's been since it last saw the player, its health, the player's health, how far away the player is, what it's doing, and one blank input that's there for a very good reason that I forget. This information is fed through a feed-forward neural net. There are five outputs, and the bot uses the highest of the five to decide where it wants to go.
Now the A* part begins. Back when the map was loaded, another global object, AStar, used a bunch of special NavMesh SceneObjects to create a grid of navigation points spanning the map. I have a good picture of this in one of my previous blog posts (a picture is worth a thousand subclauses). When the bot decides where to go, it calls one of five functions. They are stopMove(), findPathTo(), sneakUpOn(), findCoverFrom(), and wander(). The AStar object uses variations on the classic a* algorithm, and returns a list of points for the bot to follow. The bots use the original AIPlayer path following code, however keeping track of paths has been moved to c++ for simplicity's sake.
Each time the bot inflicts damage on the player, the bot's fitness score is incremented. This is how the effectiveness of each net is kept track of. When the bot dies, the AIManager::removeBot() function is called. (and a string of passive voice sentences are written) This replaces the lowest scoring net in the gene pool with the bot's net. The gene pool is then saved back to a file.
Read it again, it might make more sense this time.
Now down to the nitty-gritty description. When the game starts, the AIManager is created. It loads the neural net gene pool from a file, or creates a blank pool. (It has to be randomized from the console because of certain annoyingnesses with global object constructors.) When an AIPlayer "bot" is created, the AIManager combines two random nets from the better half of the gene pool, and gives it to the bot.
Then, the thinking begins. Each bot's thinking function is called, every other tick, by the AIManager. (I'm considering, for efficiency's sake, doing this some other way.) The bot converts all the information is uses into values from -1 to 1. There are currently 7 inputs: Whether the bot is doing anything, how long it's been since it last saw the player, its health, the player's health, how far away the player is, what it's doing, and one blank input that's there for a very good reason that I forget. This information is fed through a feed-forward neural net. There are five outputs, and the bot uses the highest of the five to decide where it wants to go.
Now the A* part begins. Back when the map was loaded, another global object, AStar, used a bunch of special NavMesh SceneObjects to create a grid of navigation points spanning the map. I have a good picture of this in one of my previous blog posts (a picture is worth a thousand subclauses). When the bot decides where to go, it calls one of five functions. They are stopMove(), findPathTo(), sneakUpOn(), findCoverFrom(), and wander(). The AStar object uses variations on the classic a* algorithm, and returns a list of points for the bot to follow. The bots use the original AIPlayer path following code, however keeping track of paths has been moved to c++ for simplicity's sake.
Each time the bot inflicts damage on the player, the bot's fitness score is incremented. This is how the effectiveness of each net is kept track of. When the bot dies, the AIManager::removeBot() function is called. (and a string of passive voice sentences are written) This replaces the lowest scoring net in the gene pool with the bot's net. The gene pool is then saved back to a file.
Read it again, it might make more sense this time.
Recent Blog Posts
| List: | 07/29/08 - Nuclear Asteroids (beta testers wanted) 07/20/08 - Summer Jobs, TGB, and Nuclear Fission 04/05/08 - A* Again (but this time it's done and you can downlad it) 03/10/08 - Complete AI System for Torque 02/07/08 - Neural Network AI 01/12/08 - When You Wish Upon A* |
|---|
Submit your own resources!| Andy Hawkins (Mar 10, 2008 at 03:18 GMT) |
| Peter Simard (Mar 10, 2008 at 03:39 GMT) |
| Morrie (Mar 10, 2008 at 03:42 GMT) |
When do you plan on releasing it?
What engine is it for TGE ot TGEA?
| Ramen-sama (Mar 10, 2008 at 04:05 GMT) |
| Jeremy Alessi (Mar 10, 2008 at 04:17 GMT) |
| Andrew Brady (Mar 10, 2008 at 04:53 GMT) |
| Craig Fortune (Mar 10, 2008 at 13:45 GMT) |
It sounds like you've gone for something very similar to what I did for my undergrad paper. Alas, I found the more I worked with NNs the less amazing I thought they were, especially in a game environment. I always thought I could have coded up the behaviours in a much better way myself - rather then relying on the NNs and GAs to do the learning.
This is not to even mention the fact that NNs are a closed box as far as debugging and getting any useful data back. Random behaviour (even guided with active supervised reinforement learning) is still random behaviour imo... who wants that? You may think random behaviour is awesome etc... but its unpredictable behaviour that you want. NNs simply don't provide this.
However, I have discussed with people before the possibilities of using NNs for complex physics functions in games... a better use imo.
| Dan Keller (Mar 10, 2008 at 19:01 GMT) |
@Morrie It's for tge, though it should port fairly easily.
@Craig One of the reasons I took the approach I did was to make random behavior essentially equivalent to unpredictable behavior. If the NN was connected directly to the bot's movement, a random net would make the bot move, well, randomly. Instead, the a* layer means the bot would charge, stand its ground, run away, etc randomly, and even a "stupid" net could be effective. Also, because these bots have no goals to accomplish besides killing the player, they don't need to be that intelligent. I've been killed several times by bots even immediately after randomizing the gene pool.
| Craig Fortune (Mar 10, 2008 at 20:31 GMT) |
You say how a "stupid" net could be effective because you use behaviours. Well isn't that just random behaviour selection if the NN isn't actually doing anything of of note that conveys a particular intelligence quality? I could do that with Math.rand()... See what I'm trying to get at here?
| Dan Keller (Mar 10, 2008 at 21:20 GMT) |
Also, haven't NNs been used successfully in games before?
| Craig Fortune (Mar 10, 2008 at 21:48 GMT) |
My point is that the fitness functions often (read: pretty much always) aren't strong enough to give properly decent feedback. Also, there is no set formula for doing a fitness function, so its just guess work. Academic research needs to go into producing a ruleset for doing game orientated fitness functions for them to be at all viable.
Yea NNs have been used in games already - so has all manner of wonky tech :)
| Maddermadcat (Mar 16, 2008 at 18:11 GMT) |
How easy will it be to implement new behaviours? Distinguishing teams, using switches and the like?
Also, you seem to have missed Ramen's question. I'd like to know that as well.
Edited on Mar 17, 2008 02:27 GMT
| Dan Keller (Mar 23, 2008 at 04:26 GMT) |
Keep in mind that neural nets are not well suited to accomplishing complex tasks in a game. I'm using them because they provide a simple, yet unpredictable, AI. So if you're making a capture the flag game or something similar, NNs are probably not for you. The a*, on the other hand, can be used in most any situation.
| Dan Keller (Mar 23, 2008 at 04:32 GMT) |
Edited on Mar 23, 2008 20:48 GMT
| Maddermadcat (Mar 26, 2008 at 20:02 GMT) |
You must be a member and be logged in to either append comments or rate this resource.
