Trees, lot of trees. (phase 2)
by Claude-Alain Fournier · 02/01/2007 (6:27 am) · 21 comments
First a teaser :
I did some performance work, and I am happy to say that the result is worth the effort. I am still playing with that 100'000 trees forest to test my new fxTreeReplicator class. See my first blog on this subject 26 January plan.
I have made a few screen shots with the FRAPS Fps, so it will give you an idea about the performance.
All the photo and some videos can be found here
First obvious fps increase is when the impact of 3d trees rendering is at the lowest, previously I was only getting about 14fps while looking at the whole forest from a distance, with the changes I made today in the code I get 27fps from same spot. So that's pretty good, considering that it's with a non optimised VC++ compiler (I don't have the pro version).
When in a low density area the fps is also quite acceptable, but it decrease with the number of 3d rendered trees. I don't have much impact on this problem as the 3d trees are rendered as tsStatic object, then it's the Torque renderer who do the work, my code is only responsible for the 2d render.
All in all I think that the performance is good and will make only little improvement on that performance from now on. I will now concentrate on a few others problems :
- When looking at trees from at certain angles it's too obvious they are billboards, so I will add a way to draw billboard according to the viewing angle. So you will see canopy when looking from above.
- Add server collision with trees, this will require some extensive changes in the collision code, I don't have the trees known by the server and surely not as shape with coll box and all, so I will have to be clever on that one. I want to have mobs and npc running into these forests so that's a very important part.
Now back to work...
I did some performance work, and I am happy to say that the result is worth the effort. I am still playing with that 100'000 trees forest to test my new fxTreeReplicator class. See my first blog on this subject 26 January plan.
I have made a few screen shots with the FRAPS Fps, so it will give you an idea about the performance.
All the photo and some videos can be found here
First obvious fps increase is when the impact of 3d trees rendering is at the lowest, previously I was only getting about 14fps while looking at the whole forest from a distance, with the changes I made today in the code I get 27fps from same spot. So that's pretty good, considering that it's with a non optimised VC++ compiler (I don't have the pro version).
When in a low density area the fps is also quite acceptable, but it decrease with the number of 3d rendered trees. I don't have much impact on this problem as the 3d trees are rendered as tsStatic object, then it's the Torque renderer who do the work, my code is only responsible for the 2d render.
All in all I think that the performance is good and will make only little improvement on that performance from now on. I will now concentrate on a few others problems :
- When looking at trees from at certain angles it's too obvious they are billboards, so I will add a way to draw billboard according to the viewing angle. So you will see canopy when looking from above.
- Add server collision with trees, this will require some extensive changes in the collision code, I don't have the trees known by the server and surely not as shape with coll box and all, so I will have to be clever on that one. I want to have mobs and npc running into these forests so that's a very important part.
Now back to work...
About the author
#2
02/01/2007 (6:45 am)
Well it's based on fxFoliageReplicator, I did not change a bit of the foliage (tree) placement code and fxFoliage Replicator do work with fxReplicatorBlocker class. So yes it's possible to create clearing, path etc..
#3
Until then Operation Flash-Point was my reference for forests.
I wonder what will be the performance on TGE.
02/01/2007 (8:19 am)
That's a blast Claude! 8-)Until then Operation Flash-Point was my reference for forests.
I wonder what will be the performance on TGE.
#4
02/01/2007 (10:02 am)
wow, that's a lot of trees ! 
#5
02/01/2007 (10:14 am)
Quote:I want to have mobs (of) npcs running into these forestsSounds like a cool game, can't wait to see it! Nice work.
#6
I have had a busy week but will be digging into those trees tomorrow morning at the latest. I'll keep you posted.
A couple of thoughts. There was a recently released resource that allowed for multiple resource in a single replicator. If you had this functionality you could have a variety of trees in one big replicator block and get a much more natural looking forest.
Also, while the massive number of trees being rendered is very impressive, you are rendering a LOT of visible distance there. You may want to develop it a little more within the context of how it is likely to be used in an actual game project, so bring in the distance fog, and cut down the over all visible distance a little. When the combination of visible distance and the point at which fog begins to kick in are use together well, you can actually create a greater sense of depth, because the fog as a tangible quality to the atmosphere. Not something you necessarily need to do now, but worth keeping in mind with regards to the possible benefits to performance in a polished demo.
I have not double checked this in the latest build of 1.5 yet, and this may not even relate to your code, but I spotted a bug in the fxShapeReplicator which caused the proportional scaling of objects to get screwed up. Don't know if you have seen that or not. But being able to tell the replicator to scale the trees proportionally within a give range, would also help add variety to the forest.
02/01/2007 (11:25 am)
Claude, looking great.I have had a busy week but will be digging into those trees tomorrow morning at the latest. I'll keep you posted.
A couple of thoughts. There was a recently released resource that allowed for multiple resource in a single replicator. If you had this functionality you could have a variety of trees in one big replicator block and get a much more natural looking forest.
Also, while the massive number of trees being rendered is very impressive, you are rendering a LOT of visible distance there. You may want to develop it a little more within the context of how it is likely to be used in an actual game project, so bring in the distance fog, and cut down the over all visible distance a little. When the combination of visible distance and the point at which fog begins to kick in are use together well, you can actually create a greater sense of depth, because the fog as a tangible quality to the atmosphere. Not something you necessarily need to do now, but worth keeping in mind with regards to the possible benefits to performance in a polished demo.
I have not double checked this in the latest build of 1.5 yet, and this may not even relate to your code, but I spotted a bug in the fxShapeReplicator which caused the proportional scaling of objects to get screwed up. Don't know if you have seen that or not. But being able to tell the replicator to scale the trees proportionally within a give range, would also help add variety to the forest.
#8
02/01/2007 (12:36 pm)
Very nice. I would love to see this working in my game. I wonder how this compares with Ben's mysterious Forest Pack? 
#9
then throw in one of those 8 sided billboard renders, and swap the texture coordinates for the billboards to get the right tree angles based on view when filling up the buffer :)
can you talk about the optimizations you've done? I've been working on my fxfoliagereplicator more lately trying to do some better batching. I posted a resource a while ago that batched them together, I'm guessing you are already doing that though to get that many rendering at that high a frame rate.
02/01/2007 (1:04 pm)
thats looking great, if you can get the color /intensity of the billboards to match your trees it'll look pretty convincing indeed.then throw in one of those 8 sided billboard renders, and swap the texture coordinates for the billboards to get the right tree angles based on view when filling up the buffer :)
can you talk about the optimizations you've done? I've been working on my fxfoliagereplicator more lately trying to do some better batching. I posted a resource a while ago that batched them together, I'm guessing you are already doing that though to get that many rendering at that high a frame rate.
#10
@ Clint, getting the billboards the match better is pretty easy, but is dependant on the lighting in any given level. So The billboards really have to be custom made for a given lighting arrangement.
I am going to work with Claude on this.
02/01/2007 (1:14 pm)
HA! That was the other thing I wanted to talk about, the 8 sided billboard thingy Clint mentioned. That would definitely help with transitions from billboard to model.@ Clint, getting the billboards the match better is pretty easy, but is dependant on the lighting in any given level. So The billboards really have to be custom made for a given lighting arrangement.
I am going to work with Claude on this.
#11
02/01/2007 (1:24 pm)
heres the resource I was talking bout, the other thing I did there was get the foliage lighting from the underlying terrain lightmap, which helped a lot to fit the foliage into the current scene.
#12
02/01/2007 (1:36 pm)
VERY COOL! Great job Clint, I didn't even know that resource existed. I was going to suggest Claude wrtie one. 
#13
Also based on what Adam said, could you send some info or a link for carving paths into fxReplicator zones please.
02/01/2007 (2:31 pm)
I believe the future of this resource is to generalise it, so that for example, one could use the code to generate 100,000 animated monsters, going from animated billboards (IFL) to animated 3d meshes. So a game like Total War or LOTR could be made with this. Clouds and volumetrics would be another application.Also based on what Adam said, could you send some info or a link for carving paths into fxReplicator zones please.
#14
It's based off the fxFoliageReplicator, I suppose you could have animated monsters and stuff, but they'll be standing in place.
Unless....you would like the nightmare of moving and orienting everything through making teeny tiny adjustments to the seed of the replicator so that you could move soldier B all the way to the front. In which case, I would build a temple in your honor, or the honor of anyone else who accomplished such.
But, there are other ways you could start with the replicator, or this resource presumably, and come out to a similar benefit as what you describe. However, I highly doubt that such will be the future of this resource.
A more practical future for this resource is to incorporate multiple shapes and billboards and include some probability and statistics information from which to determine what shape to place (dead tree, diseased tree, burnt tree, live tree, very lively tree, etc) to create an environment similar to a natural forest.
Definitely looking forward to merging my vegetation object stuff into this when it gets resourced.
02/01/2007 (5:21 pm)
@HawkinsIt's based off the fxFoliageReplicator, I suppose you could have animated monsters and stuff, but they'll be standing in place.
Unless....you would like the nightmare of moving and orienting everything through making teeny tiny adjustments to the seed of the replicator so that you could move soldier B all the way to the front. In which case, I would build a temple in your honor, or the honor of anyone else who accomplished such.
But, there are other ways you could start with the replicator, or this resource presumably, and come out to a similar benefit as what you describe. However, I highly doubt that such will be the future of this resource.
A more practical future for this resource is to incorporate multiple shapes and billboards and include some probability and statistics information from which to determine what shape to place (dead tree, diseased tree, burnt tree, live tree, very lively tree, etc) to create an environment similar to a natural forest.
Definitely looking forward to merging my vegetation object stuff into this when it gets resourced.
#15
Imposters are what you looking for:
Gamasutra article
Siggraph paper
02/01/2007 (6:26 pm)
Quote:When looking at trees from at certain angles it's too obvious they are billboards, so I will add a way to draw billboard according to the viewing angle. So you will see canopy when looking from above.
Imposters are what you looking for:
Gamasutra article
Siggraph paper
#16
Well, there are only good advice here so I may take all in consideration ;).
Let try to answer a few here :
@Stephan :
Yep Op Flashpoint is one the game a still play, just yesterday I finished again Flashpoint Resistance, I started a few weeks ago when I wanted to analyse how they do the forests and foliage, then I replayed the whole game. I am wondering if they use SpeedTree ?, but their forest are quite impressive, also from air, you realy see the switch from 2d to 3d, they also use quadtree as you see square of trees changing shapes, they also use quite a few level of LOD on trees.
@Todd :
Actually my class allow you to laydown 16 differents trees in a single forest (I use 5 in my forest here), this number can be upped or lowered in the C++ code, nothing stop you to use any other number, 1 or 432. There is one limitation though, it's the buffer size for batch rendering. The number of vertex is limited to 16384 tree in one buffer. Also I do force the visible distance to 5000 (no fog) and set the visible distance in the fxTreeReplicator to be 5000 also. There is a problem with taking the fog into consideration, this was already in the original ressource.
@Rubes :
Me too, I just whished I did not need to do this ressource, I waited for Ben forestpack for too long and our project need something now. But if Ben release his pack tomorrow, I stop working on this asap ;) >> Ben, it's a hint..... ;)
@Clint :
I will have a look at your ressource. But be aware that my code is TGEA and use shader, it's based on fxFoliageReplicator and use the same render technique, fxFoliageReplicator can show 200'000 elements as it is. My performance is mainly on how to manage the 3d shapes.
Be also aware that the culling is very important so is the level of quadtree you use, for 100'000 trees in such a big area (4km x 4km) I use a culling resolution of 1024. Using 128 reduce the fps by 10 fps. So you also need to play with culling resolution.
@Andy :
Look at this ressource : fxReplicatorBlocker
@Jonathan :
What I am currently doing here is just the beginning, I can see interest to have something like a table that tell you what kind of tree to be spawned according to altitude, ratio between trees, dead trees etc... Once I have the ressource out then I hope the community here will work on it and add such features and optimise it.
@Neo :
Thanks, interesting articles I will see if I can use this.
02/02/2007 (12:21 am)
Wow, that's a lot of interest here, and thanks all for the different links. Well, there are only good advice here so I may take all in consideration ;).
Let try to answer a few here :
@Stephan :
Yep Op Flashpoint is one the game a still play, just yesterday I finished again Flashpoint Resistance, I started a few weeks ago when I wanted to analyse how they do the forests and foliage, then I replayed the whole game. I am wondering if they use SpeedTree ?, but their forest are quite impressive, also from air, you realy see the switch from 2d to 3d, they also use quadtree as you see square of trees changing shapes, they also use quite a few level of LOD on trees.
@Todd :
Actually my class allow you to laydown 16 differents trees in a single forest (I use 5 in my forest here), this number can be upped or lowered in the C++ code, nothing stop you to use any other number, 1 or 432. There is one limitation though, it's the buffer size for batch rendering. The number of vertex is limited to 16384 tree in one buffer. Also I do force the visible distance to 5000 (no fog) and set the visible distance in the fxTreeReplicator to be 5000 also. There is a problem with taking the fog into consideration, this was already in the original ressource.
@Rubes :
Me too, I just whished I did not need to do this ressource, I waited for Ben forestpack for too long and our project need something now. But if Ben release his pack tomorrow, I stop working on this asap ;) >> Ben, it's a hint..... ;)
@Clint :
I will have a look at your ressource. But be aware that my code is TGEA and use shader, it's based on fxFoliageReplicator and use the same render technique, fxFoliageReplicator can show 200'000 elements as it is. My performance is mainly on how to manage the 3d shapes.
Be also aware that the culling is very important so is the level of quadtree you use, for 100'000 trees in such a big area (4km x 4km) I use a culling resolution of 1024. Using 128 reduce the fps by 10 fps. So you also need to play with culling resolution.
@Andy :
Look at this ressource : fxReplicatorBlocker
@Jonathan :
What I am currently doing here is just the beginning, I can see interest to have something like a table that tell you what kind of tree to be spawned according to altitude, ratio between trees, dead trees etc... Once I have the ressource out then I hope the community here will work on it and add such features and optimise it.
@Neo :
Thanks, interesting articles I will see if I can use this.
#17
I do have one general question: What does everyone else use? In regard to Near foliage (grass/plants/trees), Medium and Far Distance. Im thinking a combination of different replicators is suitable. Or would this Tree replicator handle near grass as well? Any tips? Our project is a RPG with large open world.
addikt
02/05/2007 (2:12 pm)
Im really looking foward to seeing this released as a resource. We are in dire need of a TreeReplicator for our project. Keep up the good work!I do have one general question: What does everyone else use? In regard to Near foliage (grass/plants/trees), Medium and Far Distance. Im thinking a combination of different replicators is suitable. Or would this Tree replicator handle near grass as well? Any tips? Our project is a RPG with large open world.
addikt
#18
Sorry I did not reply too soon. I think that a combination of foliage, shape and tree replicator need to be used. My solution is good only for lot of tree as the quantity make up for the quality, I said that because the switch between 2d - 3d is not perfect (this is not SpeedTree). But treeReplicator can be used for any static shapes that you won't interact with.
I also made a change in ShapeReplicator (I call it treeShapeReplicator) so it use similar technic to swith between 2d - 3d and there the shapes can be interacted with. (to be finalised for collision client & server + mechanism for interaction, like cut a tree down) the difference between TreeReplicator and TreeShapeReplicator is the rendering technique. While TreeReplicator allow you to create 100'000 tree or more in a forest, TreeShapeReplicator max out at around 2000.
02/16/2007 (12:30 am)
Addiktive,Sorry I did not reply too soon. I think that a combination of foliage, shape and tree replicator need to be used. My solution is good only for lot of tree as the quantity make up for the quality, I said that because the switch between 2d - 3d is not perfect (this is not SpeedTree). But treeReplicator can be used for any static shapes that you won't interact with.
I also made a change in ShapeReplicator (I call it treeShapeReplicator) so it use similar technic to swith between 2d - 3d and there the shapes can be interacted with. (to be finalised for collision client & server + mechanism for interaction, like cut a tree down) the difference between TreeReplicator and TreeShapeReplicator is the rendering technique. While TreeReplicator allow you to create 100'000 tree or more in a forest, TreeShapeReplicator max out at around 2000.
#19
Sorry I haven't gotten back to you on the trees yet. I have been experimenting with them, but have been get them finished this past week.
02/16/2007 (9:00 am)
Claude,Sorry I haven't gotten back to you on the trees yet. I have been experimenting with them, but have been get them finished this past week.
Torque 3D Owner Adam deGrandis
Adam deGrandis
Edit: Any plans to extend this to allow for masking or cutouts or anything? The large forest looks great, but without being able to carve out paths, its not really practical for any kind of game that you need to move through them in a focused way.
Again, though, really killer work.