The terrain engine
by Daniel Harman · in Torque Game Engine · 05/21/2004 (8:38 am) · 29 replies
Hi, new poster here and possible licensee.
I'm pretty impressed so far, but one thing that concerns me is the terrain engine. The summary says it does continuous LOD using on screen error metrics.
Are we talking about some kind of ROAM implementation etc?
If so has anyone improved on this because constantly fiddling with a vertex buffer is considered massively suboptimal these days.
More objectively, I find it really hard as a possible purchaser to get info on how to intergrate with the engine using C++ rather than scripting everything. Are there any good tutorials on this from the ground up?
I'm pretty impressed so far, but one thing that concerns me is the terrain engine. The summary says it does continuous LOD using on screen error metrics.
Are we talking about some kind of ROAM implementation etc?
If so has anyone improved on this because constantly fiddling with a vertex buffer is considered massively suboptimal these days.
More objectively, I find it really hard as a possible purchaser to get info on how to intergrate with the engine using C++ rather than scripting everything. Are there any good tutorials on this from the ground up?
#22
05/22/2004 (5:56 pm)
@Stefan: You are again correct--this model isn't nearly for all game concepts, and is in fact put together for the next couple of generations of MMOG's--ones that are fundamentally focused on game world interaction and modification. It's near impossible to have a fully interactive game world when you are limited as to where you can affect the terrain/build a building/stage a fight based on "mission boundaries". How do you tell a player "You can't build there, you would straddle a zone boundary!"?
#23
It had serious issues on block boundaries, and it kept all terrain blocks in system memory at all times. Unless you and I are thinking of very different terrain managers, it didn't support the sort of speculative loading you are talking about, although I suppose you could build such speculative loading on top of it. It didn't stream terrain data over the network.
It sounds like you're looking for a prebuilt MMO client. TSE is most assuredly not that. It will definitely be easier to do a MMO client with TSE than with TGE, but it would still represent a lot of work.
For the terrain block sizes in TSE, you're looking at about 150k of memory usage for each block, including collision data, texture data, mipped heightfield information, etc. If you have a server with 512 megs of RAM, you could comfortably store an area of terrain 40 blocks on a side. With the standard Torque terrain density, that's about 42 square kilometers per server. Given that you're hopefully not having people on every block at the same time you could take advantage of a marvelous dynamic memory management technique supported by most operating systems and store an area some 80 blocks square. This would take up about a gig of virtual memory. If you use the total memory space of your (32-bit) system, you can fit about 160 blocks square. If you give up the texture data, you can fit maybe twice that.
Of course, that's 170km square. Which is a pretty respectable area. Not as good as There, of course. But you'll notice that There isn't using all their space, either...
If you do more exotic things you can probably store big PNGs of the heightfield, dynamically load/unload areas of interest on the server side, and so forth. That can give you a lot more apparent area, but you're still limited to a certain amount of working area, and if you exceed it the results will be bad - nothing like having your MMO crash when someone tries to walk into the 161st terrain block! Or you could do server splits on the 81st block, and save yourself some hassle, but that's really well beyond the scope of what TSE is supposed to be doing (or what the terrain manager did).
Anyway, enough MMO theory.
The reason that TSE is "concerned about terrain paging"... It's not. Due to size limits on vertex buffers, we have to have a smaller block size, so a terrain manager is a necessity. It's a framework on which you could more easily build a paged terrain system but it's most certainly not going to come standard.
The big question for you, Stephan, is how you're going to fill all this area once you have it. :)
05/22/2004 (6:01 pm)
The terrain manager patch was two things. First, a cleanup of the code in Torque that accesses terrain, so that it isn't assuming a single terrainblock object. Second, a straightforward extension of the terrain logic to load and render multiple terrain blocks using Torque's terrain renderer.It had serious issues on block boundaries, and it kept all terrain blocks in system memory at all times. Unless you and I are thinking of very different terrain managers, it didn't support the sort of speculative loading you are talking about, although I suppose you could build such speculative loading on top of it. It didn't stream terrain data over the network.
It sounds like you're looking for a prebuilt MMO client. TSE is most assuredly not that. It will definitely be easier to do a MMO client with TSE than with TGE, but it would still represent a lot of work.
For the terrain block sizes in TSE, you're looking at about 150k of memory usage for each block, including collision data, texture data, mipped heightfield information, etc. If you have a server with 512 megs of RAM, you could comfortably store an area of terrain 40 blocks on a side. With the standard Torque terrain density, that's about 42 square kilometers per server. Given that you're hopefully not having people on every block at the same time you could take advantage of a marvelous dynamic memory management technique supported by most operating systems and store an area some 80 blocks square. This would take up about a gig of virtual memory. If you use the total memory space of your (32-bit) system, you can fit about 160 blocks square. If you give up the texture data, you can fit maybe twice that.
Of course, that's 170km square. Which is a pretty respectable area. Not as good as There, of course. But you'll notice that There isn't using all their space, either...
If you do more exotic things you can probably store big PNGs of the heightfield, dynamically load/unload areas of interest on the server side, and so forth. That can give you a lot more apparent area, but you're still limited to a certain amount of working area, and if you exceed it the results will be bad - nothing like having your MMO crash when someone tries to walk into the 161st terrain block! Or you could do server splits on the 81st block, and save yourself some hassle, but that's really well beyond the scope of what TSE is supposed to be doing (or what the terrain manager did).
Anyway, enough MMO theory.
The reason that TSE is "concerned about terrain paging"... It's not. Due to size limits on vertex buffers, we have to have a smaller block size, so a terrain manager is a necessity. It's a framework on which you could more easily build a paged terrain system but it's most certainly not going to come standard.
The big question for you, Stephan, is how you're going to fill all this area once you have it. :)
#24
City of heroes is zoned. No one minds. Heck, some people point out the multiple zone instances to reduce crowding as one of the selling points of the game.
05/22/2004 (6:07 pm)
Quote:You are kidding, right? Hehe..sorry to be sarcastic, but one of the SINGLE BIGGEST complaints our surveys have shown from EQ players is "I hate waiting to load a new zone".
City of heroes is zoned. No one minds. Heck, some people point out the multiple zone instances to reduce crowding as one of the selling points of the game.
#25
@Ben: One of the things I'm learning as we dig deeper into the SDK is that the "apparent" limitations of Torque Terrain aren't nearly as bad as their image. People see "a terrain block uses a 256x256 height bitmap", and posts like "just build your terrain so mission boundaries aren't obvious" and think all sorts of things that at least at the moment appear not to be true.
Regarding the Terrain Manager resource, we're saying the same thing: It's a thin API that allows you to abstract out terrain loading/paging to allow more than 1 block in the background, which gives you the ability to basically have more than one "mission" available to the client at once, to ease the transition (player perceived) wait states. You are correct, it has -huge- issues with boundary transitions, which is one of the very limiting downfalls of the design (just a challenge, not a gamestopper). Again, you are correct, it didn't directly support "speculative loading", but it went a step further than the inherent SDK does for that purpose. From my conversations with Bryan (e-mail), it was a step in the direction if his desired terrain paging system.
Regarding your memory models, you're absolutely correct--it's not actually the terrain itself that is an issue, but what is -in- the terrain that can be showstoppers, performance wise.
"It sounds like you are looking for a pre-built MMO client"--well, we're not -expecting- anything at all, that's why it's an SDK! Our design already took care of all of the topics above, and we only paused in implementation because there was a bit of confusion between what we meant by terrain paging and what you meant by terrain paging. Not a big deal at all, and it doesn't mean that as a community we cannot help said SDK to meet some new design concepts! It also doesn't in any way imply a slight on TSE--we are very much looking foward to it's release, as I imagine the rest of the community is as well.
Finally, you mentioned "how are you going to fill all that area once you have it?" This is -not- meant to be an insult, but the entire purpose behind an interactive/persistent world is NOT to provide content to the player, but to provide an environment where the player can make the content (or not, as they choose). It's not my responsibility to fill this "area"--it's my responsibility to provide a simulation that allows the player to fill this area :)
05/22/2004 (6:22 pm)
@Mark: CoH is in NO way a dynamic/persistent world. Don't get me wrong, it's an outstanding game, but it's not designed to be an environment where the seamless terrain concept is even desired, much less required.@Ben: One of the things I'm learning as we dig deeper into the SDK is that the "apparent" limitations of Torque Terrain aren't nearly as bad as their image. People see "a terrain block uses a 256x256 height bitmap", and posts like "just build your terrain so mission boundaries aren't obvious" and think all sorts of things that at least at the moment appear not to be true.
Regarding the Terrain Manager resource, we're saying the same thing: It's a thin API that allows you to abstract out terrain loading/paging to allow more than 1 block in the background, which gives you the ability to basically have more than one "mission" available to the client at once, to ease the transition (player perceived) wait states. You are correct, it has -huge- issues with boundary transitions, which is one of the very limiting downfalls of the design (just a challenge, not a gamestopper). Again, you are correct, it didn't directly support "speculative loading", but it went a step further than the inherent SDK does for that purpose. From my conversations with Bryan (e-mail), it was a step in the direction if his desired terrain paging system.
Regarding your memory models, you're absolutely correct--it's not actually the terrain itself that is an issue, but what is -in- the terrain that can be showstoppers, performance wise.
"It sounds like you are looking for a pre-built MMO client"--well, we're not -expecting- anything at all, that's why it's an SDK! Our design already took care of all of the topics above, and we only paused in implementation because there was a bit of confusion between what we meant by terrain paging and what you meant by terrain paging. Not a big deal at all, and it doesn't mean that as a community we cannot help said SDK to meet some new design concepts! It also doesn't in any way imply a slight on TSE--we are very much looking foward to it's release, as I imagine the rest of the community is as well.
Finally, you mentioned "how are you going to fill all that area once you have it?" This is -not- meant to be an insult, but the entire purpose behind an interactive/persistent world is NOT to provide content to the player, but to provide an environment where the player can make the content (or not, as they choose). It's not my responsibility to fill this "area"--it's my responsibility to provide a simulation that allows the player to fill this area :)
#26
05/22/2004 (7:31 pm)
Stephen, I don't doubt that there are designs that call for a seamless world. I was specifically answering that bit about streaming being necessary for a 'MMOG application structure'. Your MMO game might need this, but MMO games in general seem to do pretty well without it.
#27
05/22/2004 (9:13 pm)
Well spoken, Stephen. Best of luck to you.
#28
05/23/2004 (8:59 am)
Stephen, this is kind of off-topic, but I was also very gung-ho about the "sandbox" theory for MMO's back in the day. Then Ultima Online came out, and I discovered that most players aren't interested in responsibly sharing the game world with others. If you gave them free reign, it'd be anarchy online, if you excuse the pun. I'm curious how you are planning to address that, or if you are not, what your philosophy is on that.
#29
However, our underlying project goal is to provide an interactive environment, and guide the players through our game mechanics to a stable yet conflict-filled world. Shadowbane's model was similar to ours: establish things your players can do, and establish consequences that make some of those things require extensive thought (they failed at this, IMO: being errant, once you reached high level, simply didn't matter--in fact, it was beneficial to be errant. FYI, "being errant" meant there were little to no consequences in their world for whatever you did).
Nemesis Vortex will model extensive NPC populations as well as the player populations, which we feel will give an extended set of available actions, as well as consequences, for the acts of players, organizations, and empires. It will also provide a "feedback layer" which (we hope) will tend to stabilize large swings in individual player organizational power--it was relatively easy in Shadowbane (the closest commercial release to the type of world we are looking at) to cause a swing of power by influencing the entire player population (game boards, etc.), but when you factor in the npc populations as a world determinstic influence, we hope to see those swings be damped to a more predictive behaviour.
A lot of MUDS, and several online environments (There, Second Life) are demonstrating stabilized player socio-geopolitical environments, and while ultimately we don't want to control the game worlds, we do feel we can stabilize and influence through game mechanics.
05/23/2004 (8:57 pm)
Well, to be honest, it's not an area of the design that has been extended past the initial phase for our project.However, our underlying project goal is to provide an interactive environment, and guide the players through our game mechanics to a stable yet conflict-filled world. Shadowbane's model was similar to ours: establish things your players can do, and establish consequences that make some of those things require extensive thought (they failed at this, IMO: being errant, once you reached high level, simply didn't matter--in fact, it was beneficial to be errant. FYI, "being errant" meant there were little to no consequences in their world for whatever you did).
Nemesis Vortex will model extensive NPC populations as well as the player populations, which we feel will give an extended set of available actions, as well as consequences, for the acts of players, organizations, and empires. It will also provide a "feedback layer" which (we hope) will tend to stabilize large swings in individual player organizational power--it was relatively easy in Shadowbane (the closest commercial release to the type of world we are looking at) to cause a swing of power by influencing the entire player population (game boards, etc.), but when you factor in the npc populations as a world determinstic influence, we hope to see those swings be damped to a more predictive behaviour.
A lot of MUDS, and several online environments (There, Second Life) are demonstrating stabilized player socio-geopolitical environments, and while ultimately we don't want to control the game worlds, we do feel we can stabilize and influence through game mechanics.
Torque 3D Owner Stephen Zepp
The thing is, due to how money flows from the consumer to the producer to (what little is left) the developer, generational changes in technology are -very- slow to become widespread. Shadowbane for example still uses the "to hit" model of AD&D (which is technology built in the 1970's) to determine the success of an attack--armor in the real world makes you EASIER to hit (you can't move much), not harder, yet our gaming models still use this technology that thinks armor makes you harder to be hit (I know, the model and underlying assumptions are a lot more involved then my simplification, but still--it's old tech). It doesn't mean that it is the best (or even the top 10) model to use, it's just market/developer inertia.
Asheron's Call is very nearly the exact model we're talking about. AC failed for not a few reasons, but ask anyone that played for a while--user perceived lag was NOT one of it's weak points: users were -very- happy with the perceived terrain from both lag and availability (size) perspective. Shadowbane uses a (IMO poorly implemented) version of this design as well, and while lag is a BIG reason for it's (pending downfall), it's not really due to the terrain paging model.