Atlas Question

#1 I don't know what you have done to try this but Atlas definatly supports this right now, out of the box.

#2 Atlas's current incarnation supports areas as large as 10^154 chunks on a side, for a grand total of 6.5 x 10 ^ 312 chunks on the lowest level of the quadtree. For reference, the surface area of the earth is approximately 5.1 x 10 ^ 15 square meters...

The upshot of all this is you can make a terrain as big as you want. You can make a ringworld, you can make a dyson sphere (well, you could if the terrain code supported spherical terrains). The only issues are going to be how much disk space you want to burn on it, and that your working set will increase slowly as you have deeper and deeper trees. With 512x512 compressed texture tiles, a 256 MB video card should just about be able to render the maximum size tree without running out of space.

Bear in mind that each chunk represents something like a 256x256 section of heightfield; 10^154 chunks at 256 height samples on a side, with 1.0 meter sample spacing (resulting in a total height density 64 times greater than what Torque currently supports), represents an area 2.5e157 meters on a side.

For reference, Pluto's mean distance from the sun is 5.9e12 meters. So for the theoretical maximum Atlas terrain (which would still render at something approximating realtime... theoretically ;), we can fit something like 10e145 SOLAR SYSTEMS in that working set.

The big problem here is simply that you'd need a disk storage array that could handle something along the lines of 6.5e300 megabytes of data storage. If you wrote a fractally driven procedural terrain data generator you'd be able to cut the storage size down to practically nothing, and there'd you have it, a terrain with quite a lot of area.

It would be 2.64e141 lightyears on a side. You could fit approximately 1.8e128 copies of the visible universe along one side of this terrain, and you could run across it at a normal running speed and be impressed the whole way at the detailed nature of it.

For a more realistic answer to your question, you can basically a pretty detailed terrain that is approximately 8 km on a side in about a hundred megabytes (5-10 megabytes of geometry info and the rest texture data). Each increase in size-on-a-side by a factor of two will quadruple storage requirements. So if you wanted to have a 32km on a side patch of unique terrain, it'll take you about 400MB. This is 1024 unique sq km of terrain. For reference, the whole of New York City, the largest city in the US, with approximately 8 million citizens, is 1200 sq km of area, with about 30% of that water.

And this dataset would run just fine on consumer hardware; on mid-range to high-end hardware you'd have enough space left to comfortably run your game.

And if you want it to just tile forever, you can do that pretty easily now - just put a bunch of AtlasInstances in your mission. If they're all for the same datasets, the overhead is negligible. I plan on adding a tiling flag in the future; I have some other projects to get done first, though, like TGE 1.4 and TDN.

#3 *Ben flexes for the ladies*

#4 Hey, I just want to make sure people have a clear idea just how big my... er... terrain is. ;)

#5 I think my head just exploded...

:)

#6 LOL that was a rather impressive display there, Ben. :) Good info all the same.

#7

Quote:For a more realistic answer to your question, you can basically a pretty detailed terrain that is approximately 8 km on a side in about a hundred megabytes (5-10 megabytes of geometry info and the rest texture data). Each increase in size-on-a-side by a factor of two will quadruple storage requirements. So if you wanted to have a 32km on a side patch of unique terrain, it'll take you about 400MB. This is 1024 unique sq km of terrain.

I have two questions.

First of all, if doubling the size per side quadruples storage requirements, wouldn't a 32 by 32 km area require 1,600 Mb of storage rather than 400? From your description I would have thought the requirements would be as follows:-

8 by 8 (64 sq km) = 100 Mb
16 by 16 (256 sq km) = 400 Mb
32 by 32 (1,024 sq km) = 1,600 Mb (1.5625 Gb)
64 by 64 (4,096 sq km) = 6,400 Mb (6.25 Gb)
128 by 128 (16,384 sq km) = 25,600 Mb (25 Gb)
256 by 256 (65,536 sq km) = 102,400 Mb (100 Gb)

Secondly, what methods are available to reduce the storage requirements?

Ideally, we would like to be able to create a terrain a little smaller than the 65,536 sq km example, but only use as much disk space as the 256 sq km one. Could we use unique geometry, but recycle texture data somehow?

#8 *insert favorite expletive here*! Thats insane! Wow, so I can do this now... bah.

The only thing is (I still don't understand completely), is there a way to load each instance of terrain as a mission or something but still have it flow together as if it were just a bunch of Atlas instances inside one mission file. I would like to be able to put plenty of interiors and instances on each terrain block, but if I had my whole game as one mission file, all those items, interiors, and objects would take up pages and pages in the World Edit Inspector.

What would be really nice is if we could have a way to organize objects according to each terrain instance. Such as, we could have it break down according to the x and y coordinates for each Atlas instance. All of the objects and interiors inside the x and y coordinates of each terrain instance would be put inside each terrains' drop down thingy(the + and - box things, ya'll know what I'm talking about). If it was done like that, it would be very easy to have everything inside one mission file. This is probably something that has already been done too, and I'm just too dumb to realize it. This terrain is just so amazing it's turned me retarded.

#9

Quote:*Ben flexes for the ladies*

funny shit :)

Bens post should be "sticky", I believe that this will be asked alot in the future.....

#10 @Wysardry:

For the first question, I believe your math is correct. If space is a huge issue, you can probably do things like zlib compress the geometry and texture data (although this may effect runtime paging performance; if you try it let me know how it works for you!).

For the second, Atlas leaves you free to scale the detail as needed. If you're willing to have less detailed textures, you can simply keep the same TQT file size; things will simply be stretched more. If you take the time to read the Atlas documentation, you'll find I outline several deficiencies in the current implementation, as well as my plans to resolve them; I believe the runtime blender system I propose there covers your scenario nicely.

@Will:

Yes, a streaming world system would be cool. :) Currently, you'd have to create seperate missions and switch between them, or come up with your own system for loading/unloading things on the fly. A couple of people have been working on this but I've not seen any finished implementations... Just rumors. :P

#11 Ben: Unfortunately I'm currently unable to compile TSE, so cannot test the options you suggested. Knowing that there are options helps though, thanks.

I'm sorry, I did try to read the terrain documentation, but I felt my eyes glazing over long before I reached the section covering limitations. I'll try again, starting with that page first.

#12 Forgive me if this has been explained elsewhere, but I wasn't able to figure out the set of keywords if it has.

If someone does take a huge terrain, and then adds several cities and such around it (a large RPG, as Will was speaking about) would the only noticable downfall be the "pages and pages" the listing consumes in the World Editor? More specifically, if you have millions of objects (trees, buldings, npcs) scattered around -- are you going to take a massive hit when it has to decide to cull all those objects, or has the rendering system been optimized for such cases?

Obviously, managing many pages of objects at once would be a ridiculous task and one would want to implement some sort of streameing mission cells, if only to avoid that... I'm just curious as to how the environment would react before such a system was implemented.

#13 I've mentioned this elsewhere, but the terrain is what is paged, nothing else. You will need to implement some form of optimization within a very large/massive game to handle thousands/millions of objects. This type of implementation will be so specific to a game (and very complex in most cases) as to require custom implementation for your project.

#14 I think many people are wanting to develop one large world/city with thousands of buildings in it now without haveing to do any changes no matter the game. Are there any plans to do better culling of LODs or models and things to optimize this?

#15 @Kip: rendering is absolutely not the issue here--it's simply the overhead of ticking 1,000,000+ objects every tick. Proper use of LoD handles a huge portion of the render culling issues, and if it is required, you can also use gameplay mechanics (such as view blocking terrain, use of culling planes with some custom implementation of that concept, etc).

What I was implying with my post above is that simply having a hugely massive terrain is just the very first step for making a hugely massive world. Since this type of implementation is so specific to gameplay, it's just not something that will be involved in a stock implementation, although an example implementation of something that begins this type of optimizing process might make for a good resource if anyone wants to give it a shot!

#16 Ideally (the word that programmers hate most from business owners), areas out of gameplay region aren't updated at all, but "come to life" when they're needed.

I suppose that in reality, you'd have to keep track of certain things, but thats mostly dependent on how critical the state is in relation to the game. For instance, I'm doing something on one side of Ivantropolis, meanwhile on the other side of town an NPC is proceeding about 3 tasks to be completed at 3 different points in time. If there are cars driving around, they can move and honk etc, but after they go out of my view for more than 2 minutes, they can remove themselves from the tickables...

I still haven't had much time to explore TSE or TGE in depth though, curse the lure of short-term consulting :(

Congrats to GG though, this milestone is very exciting.

#17 @Ivan

Quote:
Ideally (the word that programmers hate most from business owners), areas out of gameplay region aren't updated at all, but "come to life" when they're needed.

That would generally be the idea behind streaming worlds. The tricky part is make it look like time has past if the user reenters the area. it would be unrealistic if you left ,for instance, a battle and came back days later and found the battle the same as when you left it.

Quote:If there are cars driving around, they can move and honk etc, but after they go out of my view for more than 2 minutes, they can remove themselves from the tickables...

I think in that case, you would just destroy the car once it leave your viewing range. That how they did it with GTA3 .

#18 Interesting story: Mark F and I had a discussion about this exact issue over beers one night, and we started talking about using quantum principles, and defining some functions that would emulate sum over histories to find the "most probable" re-entrant state when an observer goes into an area that hasn't been observed in quite a while (and therefore needs updates).

Obviously, sum over histories isn't gonna work on your generic MMOG server or anything, but the principle is the same: have a function (mathematical I mean, not syntactic) that can take the last known timestamp when a particular area was observed, and a heuristic for bringing it up to speed quickly.

Alternatively, you can use what I call a priority queue system, where you basically tick areas of control that are currently observed every tick, while areas that can be predicted to not be observed any time soon at higher and higher ratios of "update ticks" to "real ticks".

In other words, if an area hasn't seen a human player for 20 minutes, it might be set to only get ticks 1 for every 100, while an area that hasn't been seen in weeks would be getting ticks say 1 for every 10,000 or so--and your updating algorithm would be able to handle differential tick periods.

#19 It might be worth looking into how the Elder Scrolls series of games handle this.

Daggerfall (TES II) had a continuous playable area of around 100,000 square miles (259,000 sq km), which was "paged in" as needed, although interiors were loaded separately. Unfortunately, I haven't seen any information on how they managed this on a 486 with 8Mb RAM.

Morrowind (TES III) had a much smaller playable area (around 10 square miles), which was more detailed. In the settings/preferences menues, there is an option to change the LOD for NPC AI updates, and those further away are updated less frequently, so I would imagine it uses a method similar to those you described.

Oblivion (TES IV) is rumoured to have a playable area of around 40 square miles (it is still being developed). The devs have mentioned that there will be over 1000 dynamic NPCs whose actions are based upon needs and/or schedules using Radiant AI.

As for avoiding a huge list of objects in the editor, would it not be possible to build that based upon a LOD/distance from camera setting?

Most of the time you wouldn't need to list objects that are too far away to be seen, and when you did you could increase the detail level. Personally, I would prefer to move the camera instead, so that I could be sure I was really editing the object I meant to (rather than one with a similar name).

Alternatively, it could be based upon which terrain "chunk" is selected.

#20 A very large portion of Daggerfall has the content randomly generated on demand. I'm not sure how 3/4 worked, but there is a writeup on the internet about how Dungeon Siege handled theirs...try searching for Dungeon Siege and polling through the results.