Opening discussion on Terrain Paging
by David Dougher · in Torque Game Engine · 06/02/2004 (2:48 pm) · 56 replies
This thread will be used to discuss Terrain Paging.
First of all acknowledgements are in order.
J. Donavan Stanley did most of the updates to the Terrain Manager after its initial creation those changes became the project known as Gorpe.
Fredrik Svensson did a great deal of work on my first iterations of the Terrain Paging. Although the final version of the code is mine he provided a lot of useful feedback and helped with much of the hashing that goes into working out everything that was being touched and where.
Briefly then,
My Implementation is quite simple. It assumes a 3x3 tile system with the player at the center. As the player moves from one boundary to the next the appropriate tiles are loaded and the tiles furthest from the player are dropped.
Worst case scenario is moving at a diagonal from corner tile to corner tile. This causes the loss of five tiles and the loading of five.
A couple of simple checks are run to determine which tile the player is on and entering a new tile initiates a page check. Depending on which direction you are moving the further tiles are let go. The the new tiles are loaded. There are special tests for edge conditions and corners.
Essentially, if you run to an edge or a corner of the world the tiles you currently have are retained and you move off of the center tile. The reason for this is simplicity - it is easier to maintain the 3x3 matrix then to special case all the differences. Also, if you run into a corner you will still have to come back out -- meaning that we have already cached the tiles you will run to eventually when leaving.
As you return from a corner or edge we move you back the center tile.
First of all acknowledgements are in order.
J. Donavan Stanley did most of the updates to the Terrain Manager after its initial creation those changes became the project known as Gorpe.
Fredrik Svensson did a great deal of work on my first iterations of the Terrain Paging. Although the final version of the code is mine he provided a lot of useful feedback and helped with much of the hashing that goes into working out everything that was being touched and where.
Briefly then,
My Implementation is quite simple. It assumes a 3x3 tile system with the player at the center. As the player moves from one boundary to the next the appropriate tiles are loaded and the tiles furthest from the player are dropped.
Worst case scenario is moving at a diagonal from corner tile to corner tile. This causes the loss of five tiles and the loading of five.
A couple of simple checks are run to determine which tile the player is on and entering a new tile initiates a page check. Depending on which direction you are moving the further tiles are let go. The the new tiles are loaded. There are special tests for edge conditions and corners.
Essentially, if you run to an edge or a corner of the world the tiles you currently have are retained and you move off of the center tile. The reason for this is simplicity - it is easier to maintain the 3x3 matrix then to special case all the differences. Also, if you run into a corner you will still have to come back out -- meaning that we have already cached the tiles you will run to eventually when leaving.
As you return from a corner or edge we move you back the center tile.
About the author
Owner - Pariah Games, Adjunct Professor - Bristol Community College, Mentor - Game Design - Met School Newport, Mentor - Game Design - Met School Providence
#42
btw, one can build a proto MMO for demonstration purposes or even small numbers of players, say < 1000 for the cost of 3 reasonable motherboards. It all depends on what you're doing.
If one wants to go "High end" right out of the gate that will take a lot of money, but one can still build something functional for a lot less, IMO...
And maybe we should figure out how to make sure it gets cheaper and easier...
Jay
11/21/2004 (10:20 am)
Any update on when your pack may be availalable?btw, one can build a proto MMO for demonstration purposes or even small numbers of players, say < 1000 for the cost of 3 reasonable motherboards. It all depends on what you're doing.
If one wants to go "High end" right out of the gate that will take a lot of money, but one can still build something functional for a lot less, IMO...
And maybe we should figure out how to make sure it gets cheaper and easier...
Jay
#43
Safe bet is that there will be a few things we will want to fix up... ;)
11/21/2004 (10:30 am)
Probably won't even look at isolating the components to make available until after New Years. We are running tests on it right now and it makes little sense to put up what we have when it hasn't even been verified to work yet.Safe bet is that there will be a few things we will want to fix up... ;)
#44
11/21/2004 (10:31 am)
@John: to connect the dots for you, David here is the one that gave me the old version of the tpager (which he describes above), and I'm working on the port of it from whatever code version it was against (couple of years ago) to current 1.3.
#45
Jay
11/21/2004 (11:12 am)
Ah! I see. Well, I will just keep on with what I have in the mean time.Jay
#46
11/21/2004 (11:35 pm)
I'm also working on some very scalable MMO stuff which may or may not go out with David's pack depending on timing etc.. At the moment I've got a highly scalable persistent world database system and am working on a system for ancillary MMO stuff like login handling, patch distribution, automatic handoffs between servers, global chat, character creation, inventory and server management. There's still a fair bit to do though.
#47
I have the rudiments of a master server that tracks game server(clusters at some point), login and patch servers, using skinned UI.
I stopped work on that to focus on learning the art path into TGE.
Sounds like you guys really have something cooking!
11/22/2004 (8:07 am)
Hmmm it seems like I have headed down a trail that paralells yours.I have the rudiments of a master server that tracks game server(clusters at some point), login and patch servers, using skinned UI.
I stopped work on that to focus on learning the art path into TGE.
Sounds like you guys really have something cooking!
#48
Savegame files are for the birds. If you keep track of everything in a server-side database and keep local data copies for the area the player inhabits on the client your data can be streamed down all together - terrain, placed objects, that old sandwich Bill left near that tree in the park....
DB programming is boring, I know. But you could use it to your advantage. Game crashes, just fire it back up - everything is right where it was thanks to the database. And most DBMS' have plenty of security so they can assist in preventing spoofing, etc.
Oh, I know - not the end-all be-all, just don't dismiss it out of hand because it's the most boring field in programming.... I hate it myself.
I want to do something like someone mentioned earlier - create a Morrowind-like game with multiplayer persistence. That would be cool - up to 128 players, and set up so you can join a "faction" during the game. Become a bandit and lead AI bandits on raids. Join a town militia and lead the AI guardsmen to stop the bandits....
Ok, just babbling - but the whole terrain streaming thing is what I feel is needed for this sort of game, and the DB back-end is, in my mind, an ideal compliment to it. All of your placeable items, buildings, critters, etc can be held in the DB - heck, even the terrain data for that chunk in a local DB.
Ok, sorry - just got a little excited....
Rich
12/04/2004 (7:02 am)
Actually, what I don't see mentioned anywhere near often enough when talking about persistent worlds is databases.Savegame files are for the birds. If you keep track of everything in a server-side database and keep local data copies for the area the player inhabits on the client your data can be streamed down all together - terrain, placed objects, that old sandwich Bill left near that tree in the park....
DB programming is boring, I know. But you could use it to your advantage. Game crashes, just fire it back up - everything is right where it was thanks to the database. And most DBMS' have plenty of security so they can assist in preventing spoofing, etc.
Oh, I know - not the end-all be-all, just don't dismiss it out of hand because it's the most boring field in programming.... I hate it myself.
I want to do something like someone mentioned earlier - create a Morrowind-like game with multiplayer persistence. That would be cool - up to 128 players, and set up so you can join a "faction" during the game. Become a bandit and lead AI bandits on raids. Join a town militia and lead the AI guardsmen to stop the bandits....
Ok, just babbling - but the whole terrain streaming thing is what I feel is needed for this sort of game, and the DB back-end is, in my mind, an ideal compliment to it. All of your placeable items, buildings, critters, etc can be held in the DB - heck, even the terrain data for that chunk in a local DB.
Ok, sorry - just got a little excited....
Rich
#49
In my own project's implementation of terrain paging, we'll of course wind up getting completely rid of mission files entirely, since the persistent world will be built up via database states on any particular server's bootup.
You do need to keep in mind however a distributed network's limitations--when the client first logs in for the day, you don't want to be sending him the database current values for each and every data element in the persistent world--you're talking 2 hours in some cases from click to play to being in game.
Delivering things like terrain page files, and other "mostly static" data elements during installation/pre-play phases helps with this.
12/04/2004 (7:10 am)
@Rich, you are absolutely correct, but db implementation is completely project dependent. Since mission files and terrain files are all we really have as a common reference for a resoure/pack/starter kit, my work on the terrain pager at least is factored against how "stock" does it.In my own project's implementation of terrain paging, we'll of course wind up getting completely rid of mission files entirely, since the persistent world will be built up via database states on any particular server's bootup.
You do need to keep in mind however a distributed network's limitations--when the client first logs in for the day, you don't want to be sending him the database current values for each and every data element in the persistent world--you're talking 2 hours in some cases from click to play to being in game.
Delivering things like terrain page files, and other "mostly static" data elements during installation/pre-play phases helps with this.
#50
We have many part already on test phase : login server, character creation, inventory management, patch server, world objects persistence. All working with back-end DB on the server.
I could also only agree with you, for the DB implementation being boring, I did it ;) But that's a necessary step specially if you try to make a mmorpg and I think too many people forget that.
12/04/2004 (9:39 am)
@Richard, What David did not say is that Adellion is already doing exactly what you said in your post, a part from the terrain that is not stored on the DB. Otherwise all objects in game (non static) are stored and spawned from DB. We have many part already on test phase : login server, character creation, inventory management, patch server, world objects persistence. All working with back-end DB on the server.
I could also only agree with you, for the DB implementation being boring, I did it ;) But that's a necessary step specially if you try to make a mmorpg and I think too many people forget that.
#51
12/04/2004 (9:44 am)
@Stephen, wait to see the pack David talk about. You will see that part of DB implementation can be generic. Of course each project has it's own need but if you have the basic db structure layout for you then it's easyer for peoples to build on it. DB can be very tricky, I mean with MOG you need performance on the DB side if you don't want your server doing exclusively db queries. If you have already a generic structure that offer that performance then you may avoid lot of troubles for later.
#52
12/04/2004 (10:03 am)
@Claude--yes, I'm aware of what David has planned--all I meant for my portion of the project (getting tpager working or close against 1.3) is that it won't be db focused--our db implementation I know is different from what David probably has planned for the big MMOG pack.
#53
@Claude-Alain, I wasn't aware Adellion already had login server, character creation, inventory management and patch servers. It's a bit of a shame that I'm currently duplicating a lot of that effort. Seems like some kind of miscommunication somewhere.
12/04/2004 (1:02 pm)
@Richard, I also have a database-based persistent store. My implementation uses an intermediate "world" server which lives between the torque servers and the database. The backend of the world server uses MySQL to store the objects in a generic format. This doesn't require any special table structure for each type of object - all objects are stored essentially as a set of key/value pairs (with some optimisations for speed and space). As the server runs it progressively checkpoints to the world server every few seconds. That way if there's a torque server crash a new server can be brought up and things can continue with very little interruption. The plan is that my session server (when written) will automatically reroute users to the new server and they'll experience relatively little interruption on a crash.@Claude-Alain, I wasn't aware Adellion already had login server, character creation, inventory management and patch servers. It's a bit of a shame that I'm currently duplicating a lot of that effort. Seems like some kind of miscommunication somewhere.
#54
12/04/2004 (1:26 pm)
@Zik, Well it has all, of course as David say it's all on test phase now for Adellion and it's not yet decided which feature will go into a mmorpg pack so don't be too excited. Also some feature have been written after David announced the mog pack idea so he could not have talked about it, no miscommunication at all.
#55
Any updates on the Terrain paging/ manager?
Jay
12/28/2004 (6:11 am)
I am aiming at somthing like the data store of DAOC, backpacks, crafting skills, professions etc, most of which can be genericized.Any updates on the Terrain paging/ manager?
Jay
#56
http://www.garagegames.com/mg/forums/result.thread.php?qt=24552
01/03/2005 (8:16 am)
Part of this discussion has been taken over here...http://www.garagegames.com/mg/forums/result.thread.php?qt=24552
Torque Owner David Dougher
Pariah Games
But I agree in another way. I think what many of you really want is to be able to create massive worlds and then grow it into something profitable. That is what the MOG kit would do.
The MOG Pack would let you build big worlds, create the server services needed to administer such a world, and then, when the time is right, start them up on a single system. As the population grows the services would be moved to supporting servers - lessening the load on the original server and allowing it to handle additional traffic.
This system could be done as a single large world with all objects present for smaller populations, then with zone cities and dungeons for larger worlds, and finally as massive zoned lands with Internet links. So different people with similar visions could link their game worlds together (or if you have the hardware and bandwidth) you could expand into the land of shards and multiple servers).
You would never get to hundreds of thousands of players, but like the older MUD games you could get enough players to make a fun game that would have a life of its own.
Smart "world builders" would start with a smaller number of tiles - say 16 in a 4x4 matrix - and build the world - concentrating on making it an interesting place. As the population grows you patch the game world adding new tiles. You could grow the land somewhat irregularly. 5x4 as a new region to the North is opened. 6x4 as a new region to the south is discovered thru a mountain pass. A big push westward could grow the world to 6x5 or 6x6. At one point in a test Adellion was running with 32x10 tiles. Took forever to load but it did run!
The goal would be to grow the world as population pressures increase so you don't have the issue of needing to make vast areas of content before you can open. Players can always consume content faster than you can create it, so content should not be a driving force in what you make. Gameplay and fun can be had in 1 single tile - if you make two tiles it should be because that makes it more fun.
But I digress.
If people are interested in such a Pack I will put it on the development schedule.
My current thinking is that if people want to contribute to its development I will share what we have done with the contributors for free. As for the price, when it is done - I'm going to wait and see what GG charges for the RTS pack (which I intend to grab right away) and let that serve as a guide. And we will continue to make pieces of what we have done available to the community as resources. The value of the Pack will be that it ties things together and documents how they work and how to modify it. There are far too many resources in the GG library to ignore all that good work, and I see no reason not to share credit as well. So if you want to reinvent the wheel I expect that the key pieces will be in resources on the GG site, or if you want to jump start your efforts you can buy the Pack and run with it.
Adellion's development efforts will need to get a bit farther along and we will be moving into some testing soon that will consume a lot of my time but there are clearly areas that people can work on. So don't expect this under your Christmas tree, but you might consider looking for a lot more discussion and progress reports in the new year.
I will begin to talk about this more in .Plan files since people seem to think it is worth doing. I normally don't write them up much, but in this case since it is going to be a while in development and the community is following it, it makes sense to let others track what it happening.