Where I want to go with T3D...
by Demolishun · 08/27/2012 (6:11 pm) · 7 comments
I have been thinking long and hard about where I want to take the engine for my current project. I know there is a limit to the resources to make this happen. I don't have an infinite amount of time. So I decided there are a couple of specific things that I want to make happen.
Procedural Environments
This is what I am currently working on with T3D. The goal is to completely generate scenic environments using: random seed data, height maps from any source, day of the year, elevation, and climate. This is very important to me. Number one it takes a lot of time to create decent missions. If I am going to keep my sanity and reduce development costs I need to have some tech working for me. I want the computer to do the work as much as possible. The result of this effort will be: a nice mission generator with the ability to span multiple terrain blocks (the blocks need not be in the same mission at the same time, but can be), a tool for basing missions that need hand work done to them, and a way to keep the game fresh by letting people start a completely new mission that is beautiful to "just be" in. This last result is really important to my game concept.
Modular Engine Components
I remember back in the TGE days that I really disliked the class hierarchy system for game objects. I ran into a lot of problems with this structure. When I started working with T3D I realized that a lot of the issues I had were fixed. However it still had the monolithic hierarchy that I disliked. I always thought that the ability to add functions to classes from proven modular classes would have been better and more flexible. Then a while ago someone mentioned component systems. It finally dawned on me that is what I wanted all along. Some other really smart programmers were already solving this issue. Consequently I plan on working on this as well. If I can start with some existing component system I will. I not I will bite the bullet and learn what it takes to build one. I will probably model it in Python and then convert to C++. Python is so much better for creating fast prototypes IMHO.
These are going to be the two major focus areas for my development efforts with T3D. I also plan on sharing these systems once they are developed. I believe doing so will actually make the systems better than turning them into a side product. This is something I saw with extending Python with T3D. People jumped in and helped me fix the code and the resource itself. One "Guy" was a great help when he gave me some minimal startup scripts for testing the interface and helped me fix the code. This sharing of effort made it much more stable and useful.
So that is where I am going the next few months. This is going to be my contribution to the engine and the community. If this community is going to grow, then I need to grow myself. There is already a party here, lets get it rocking so much they call the police!
Procedural Environments
This is what I am currently working on with T3D. The goal is to completely generate scenic environments using: random seed data, height maps from any source, day of the year, elevation, and climate. This is very important to me. Number one it takes a lot of time to create decent missions. If I am going to keep my sanity and reduce development costs I need to have some tech working for me. I want the computer to do the work as much as possible. The result of this effort will be: a nice mission generator with the ability to span multiple terrain blocks (the blocks need not be in the same mission at the same time, but can be), a tool for basing missions that need hand work done to them, and a way to keep the game fresh by letting people start a completely new mission that is beautiful to "just be" in. This last result is really important to my game concept.
Modular Engine Components
I remember back in the TGE days that I really disliked the class hierarchy system for game objects. I ran into a lot of problems with this structure. When I started working with T3D I realized that a lot of the issues I had were fixed. However it still had the monolithic hierarchy that I disliked. I always thought that the ability to add functions to classes from proven modular classes would have been better and more flexible. Then a while ago someone mentioned component systems. It finally dawned on me that is what I wanted all along. Some other really smart programmers were already solving this issue. Consequently I plan on working on this as well. If I can start with some existing component system I will. I not I will bite the bullet and learn what it takes to build one. I will probably model it in Python and then convert to C++. Python is so much better for creating fast prototypes IMHO.
These are going to be the two major focus areas for my development efforts with T3D. I also plan on sharing these systems once they are developed. I believe doing so will actually make the systems better than turning them into a side product. This is something I saw with extending Python with T3D. People jumped in and helped me fix the code and the resource itself. One "Guy" was a great help when he gave me some minimal startup scripts for testing the interface and helped me fix the code. This sharing of effort made it much more stable and useful.
So that is where I am going the next few months. This is going to be my contribution to the engine and the community. If this community is going to grow, then I need to grow myself. There is already a party here, lets get it rocking so much they call the police!
About the author
I love programming, I love programming things that go click, whirr, boom. For organized T3D Links visit: http://demolishun.com/?page_id=67
#2
Looks to me as if you both are soloing down a redundant road, and I bet you guys could benefit from sharing notes and experiences ;)
08/27/2012 (11:59 pm)
Frank, I've been following the other thread as well and look forward to the continued newsstream regarding this topic. Funny as it may be though, I noticed Konrad Kiss posted something similar on his FB wall yesterday.Looks to me as if you both are soloing down a redundant road, and I bet you guys could benefit from sharing notes and experiences ;)
#3
At first I went with seeded, deterministic generation, but the procedure to generate some of the content at runtime made everything too slow. So instead, I decided to generate at first run and cache. Eventually, I realized that some of the generated terrain had props blocking critical roads (we had A* routines to make sure that didn't happen, but it looks like there was a loophole somewhere) so I had to dump over 4k generated maps and decided to start all over again. This time, however, I wanted to not use Torque for the generation itself, because the start-up time of the engine was also adding up to the time it would take to generate a single zone.
Right now, I'm wrapping up a new system for our game that uses a batch of Photoshop and PHP scripts to generate game-ready .ter and .forest files. The generator runs for about 50-80 secs per zone.
I first created something that would look like a believable terrain in Photoshop and began marking down steps and settings that would work well for this. Eventually, filters like difference clouds, smart blur and crystallize (and many others) have proven to be much better than most of the generator tools (built-in or standalone) I had, so I stuck with this. My primary goal was playability and not realistic hydraulic erosion, and for that I think it worked great.
I created wrapper functions in JavaScript that I'd be able to call to automate the entire process with randomized parameters here and there. From then on it was relatively easy to do just about anything I wanted to - canyons, erosion, flora, elevated borders, etc...
It's still a w.i.p, but I'm getting closer to finalizing it. It already is able to export Torque-ready .ter and .forest files, although the terrain materials are hard-wired and the .forest file relies on generated ForestDataItem datablocks based on contents of a specific game directory. So it's somewhat tailored to our game's directory structure.
For this reason I haven't yet been able to actually see it in-game, because... well, getting a zone into Xenocell is unfortunately not as easy as it used to be with vanilla Torque. It needs to first be uploaded into the Amazon cloud where all servers and all clients are able to access it. The real problem is the caching mechanism that Amazon uses to propagate the files all over the world to all CloudFront distribution points (so it is a super-fast download everywhere). All these points need to be informed that the old data is stale (a process referred to as "invalidation") so the entire task of generating and then testing a zone is pretty tedious, and I didn't feel like it was worth the time to automate it.
But I digress. Here's the image Christian mentioned in his post:
08/28/2012 (3:48 am)
Procedural content generation is really, really awesome! I've been researching it for years now. In Xenocell we have groups of settings that we refer to as an ecosystem. It includes settings for terrain generation, ground cover, props, forestdata items, terrain materials, etc... Most of the things you see in a mission file are generated procedurally based on a number of settings.At first I went with seeded, deterministic generation, but the procedure to generate some of the content at runtime made everything too slow. So instead, I decided to generate at first run and cache. Eventually, I realized that some of the generated terrain had props blocking critical roads (we had A* routines to make sure that didn't happen, but it looks like there was a loophole somewhere) so I had to dump over 4k generated maps and decided to start all over again. This time, however, I wanted to not use Torque for the generation itself, because the start-up time of the engine was also adding up to the time it would take to generate a single zone.
Right now, I'm wrapping up a new system for our game that uses a batch of Photoshop and PHP scripts to generate game-ready .ter and .forest files. The generator runs for about 50-80 secs per zone.
I first created something that would look like a believable terrain in Photoshop and began marking down steps and settings that would work well for this. Eventually, filters like difference clouds, smart blur and crystallize (and many others) have proven to be much better than most of the generator tools (built-in or standalone) I had, so I stuck with this. My primary goal was playability and not realistic hydraulic erosion, and for that I think it worked great.
I created wrapper functions in JavaScript that I'd be able to call to automate the entire process with randomized parameters here and there. From then on it was relatively easy to do just about anything I wanted to - canyons, erosion, flora, elevated borders, etc...
It's still a w.i.p, but I'm getting closer to finalizing it. It already is able to export Torque-ready .ter and .forest files, although the terrain materials are hard-wired and the .forest file relies on generated ForestDataItem datablocks based on contents of a specific game directory. So it's somewhat tailored to our game's directory structure.
For this reason I haven't yet been able to actually see it in-game, because... well, getting a zone into Xenocell is unfortunately not as easy as it used to be with vanilla Torque. It needs to first be uploaded into the Amazon cloud where all servers and all clients are able to access it. The real problem is the caching mechanism that Amazon uses to propagate the files all over the world to all CloudFront distribution points (so it is a super-fast download everywhere). All these points need to be informed that the old data is stale (a process referred to as "invalidation") so the entire task of generating and then testing a zone is pretty tedious, and I didn't feel like it was worth the time to automate it.
But I digress. Here's the image Christian mentioned in his post:
#4
08/28/2012 (6:11 am)
What Konrad said! :)
#5
That looks great.
Edit:
@Konrad,
Thanks for posting the info about using a bitmap manipulation program. I had been thinking about that, but was having a lot of fun playing around with console commands. I did a quick check and looked up ImageMagick license. It allows one to include it in a commercial application with a small attribution requirement. So potentially it could be used for bitmap manipulation using some advanced image creation/modification tools. I know I have played around with using it to create dynamic images for websites.
I think my next step is to use my Python interface. It already has bindings to ImageMagick. So any resource I create for terrain generation will require the Python resource. This is the quickest path to accomplish this for me. Also, it is much easier to thread the process to keep the game running smoothly. I looked at what it takes to write a .ter file and it is simple. Thanks for the heads up on Photoshop. That is a great solution!
08/28/2012 (8:46 am)
@Konrad,That looks great.
Edit:
@Konrad,
Thanks for posting the info about using a bitmap manipulation program. I had been thinking about that, but was having a lot of fun playing around with console commands. I did a quick check and looked up ImageMagick license. It allows one to include it in a commercial application with a small attribution requirement. So potentially it could be used for bitmap manipulation using some advanced image creation/modification tools. I know I have played around with using it to create dynamic images for websites.
I think my next step is to use my Python interface. It already has bindings to ImageMagick. So any resource I create for terrain generation will require the Python resource. This is the quickest path to accomplish this for me. Also, it is much easier to thread the process to keep the game running smoothly. I looked at what it takes to write a .ter file and it is simple. Thanks for the heads up on Photoshop. That is a great solution!
#6
If the reason is the latter of these two, then I'm not entirely sure a 100% procedurally generated terrain is actually the 'best' option.
I'm actually of a mind to have a bunch of maps (height, diffuse and detail mask maps) to call from depending on certain criteria (such as climate, latitude, etc.,). Effectively you'd have an entirely new library within the engine, if I'm seeing this right. Sure, you could have the heightmaps generated procedurally, but not at runtime. This would also allow you to populate the terrain with forest and ground cover objects depending on the above criteria, and looking something akin to what Konrad has up there.
I have issues with this kind of approach however, as it lacks a top-down design, or 'hand of God' feel to the level. Sure, you'll get the nice path with cliffs lined with trees which you can run (or walk) around in, but it'll lack that magical sense of place that a well thought out and sculpted map can - and should - deliver.
Still, if your goal is to procedurally generate your maps because you love that concept, then I say good luck. It's doable and you should learn quite a lot from the entire process, and that's a worthy endeavour, no matter what my opinions on the matter are.
As for the second of your goals, I have NFI what you're talking about. :D
08/28/2012 (9:22 am)
Why do the terrains need to be procedurally generated, Frank? Is it because you really want it to be unique for each client, or is this merely the solution you think is best for generating good terrains to work with in the shortest amount of time?If the reason is the latter of these two, then I'm not entirely sure a 100% procedurally generated terrain is actually the 'best' option.
I'm actually of a mind to have a bunch of maps (height, diffuse and detail mask maps) to call from depending on certain criteria (such as climate, latitude, etc.,). Effectively you'd have an entirely new library within the engine, if I'm seeing this right. Sure, you could have the heightmaps generated procedurally, but not at runtime. This would also allow you to populate the terrain with forest and ground cover objects depending on the above criteria, and looking something akin to what Konrad has up there.
I have issues with this kind of approach however, as it lacks a top-down design, or 'hand of God' feel to the level. Sure, you'll get the nice path with cliffs lined with trees which you can run (or walk) around in, but it'll lack that magical sense of place that a well thought out and sculpted map can - and should - deliver.
Still, if your goal is to procedurally generate your maps because you love that concept, then I say good luck. It's doable and you should learn quite a lot from the entire process, and that's a worthy endeavour, no matter what my opinions on the matter are.
As for the second of your goals, I have NFI what you're talking about. :D
#7
I am hoping to get a nice natural feel to the levels. I really liked your blog about simulating natural processes to create beautiful terrains. I had planned on taking a multi-pass approach.
Now, this will take a lot of experimenting to figure out what order and what to repeat to get a good looking terrain. It may be different for one area versus another. If I run this in another thread (like in a Python thread) it could be done at run time without bogging down the game. I even have the option of putting this in another process using the same thread type interface. So it could easily be backgrounded.
There will be lots of other uses too. Like if you had a beautifully sculpted landscape you could literally stamp changes into the level and move things around to simulate time passage or an event. Usually for stuff like that you would have to create a new .ter file or mission. Where it could literally be an extra bitmap to apply those changes.
08/28/2012 (5:16 pm)
@Dan,I am hoping to get a nice natural feel to the levels. I really liked your blog about simulating natural processes to create beautiful terrains. I had planned on taking a multi-pass approach.
- Do a base noise pass with some variation in height and turbulence.
- Throw some upheaval like strikes, plate movements, etc.
- Put in water by further splitting things using a fractal branch algorithm.
- Use some weathering to simulate erosion. Possibly generate a hardness map to keep the erosion from being uniform.
- Use normals and height to determine texture and vegetation placement.
- Place objects and supporting structures. Create an algorithm that looks for easiest traversal path for roads and passage ways.
Now, this will take a lot of experimenting to figure out what order and what to repeat to get a good looking terrain. It may be different for one area versus another. If I run this in another thread (like in a Python thread) it could be done at run time without bogging down the game. I even have the option of putting this in another process using the same thread type interface. So it could easily be backgrounded.
There will be lots of other uses too. Like if you had a beautifully sculpted landscape you could literally stamp changes into the level and move things around to simulate time passage or an event. Usually for stuff like that you would have to create a new .ter file or mission. Where it could literally be an extra bitmap to apply those changes.
Torque Owner 000
-
Sounds like an interesting project, I look forward to seeing more in the future!