fxFoliageReplicator Christmas Update
by Clint S. Brewer · 12/28/2004 (2:44 pm) · 52 comments
Download Code File
original submission, see edits at end of article
I've been working on fxFoliage a bit for my current project:
www.unearthedgames.com/
This should be a drop in replacement that improves a bit on the great work Melv started, and adds some of the work from Bendik Stang as well as a few new things.
My personal favorite is that the foliage now uses the terrains lightmap so you don't have very bright grass in shadowed areas.
To use this, it should just be a drop in, the only thing you will definitely have to change is the script where the foliage replication is started, now that it uses the terrain lighting, this must be done after the scene is lit. See the notes at the top of the .cc file for details.
Here they are along with the list of changes and things I would like see done:
and here are a couple fxFoliage data blocks from my mission file I was using to test with:
edit 01/11/2005
- Added support for cacheing foliage to .mfr files. Similar to .ml lighting cache files.
It's turned off by default, but you can set
$pref::foliage::cacheFoliage =1;
to activate it. It's turned off by default because the files it generates can be large( even with zip compression ) and there is no support for automatically deleting old or invalid files.
- Used a DataChunker to allocate the foliage items, should result in better performance as foliage item memory is allocated together.
- Cleaned up the foliage console output, added a nice summary and printout of total foliage replication time.
I had hoped to save a lot of foliage generation time and at first I had, but in looking at things more closely it seems that foliage generation time wasn't that bad in the first place, it's other object creation that seems to be slowing my own mission down. have to look into that.
When using cacheing the first time, foliage generation is a lot slower since it has to write about a 30 meg compressed file for one of my scenes, but when loading a cached file it only takes about half the time of the original. So it's a real tradeoff in space vs loading time, hence the default to not caching.
After using the DataChunker to allocate memory everything felt faster, but actual benchmarking results were inconclusive you try and let me know :)
original submission, see edits at end of article
I've been working on fxFoliage a bit for my current project:
www.unearthedgames.com/
This should be a drop in replacement that improves a bit on the great work Melv started, and adds some of the work from Bendik Stang as well as a few new things.
My personal favorite is that the foliage now uses the terrains lightmap so you don't have very bright grass in shadowed areas.
To use this, it should just be a drop in, the only thing you will definitely have to change is the script where the foliage replication is started, now that it uses the terrain lighting, this must be done after the scene is lit. See the notes at the top of the .cc file for details.
Here they are along with the list of changes and things I would like see done:
// a Big Thanks to Melv!, and also thanks to Bendik Stang! for his work on fxGrass
// of which many of the updates to fxFoliage are based.
// - Clint S. Brewer
//
// here are the changes I've made 12-28-2004 clint
// - Foliage items now use the terrain lightmap for their ambient color
// see notes in "Installation into Torque instructions" below
// - Optionally Shrink on fade, like fxGrass does
// - Optionally Align to face the camera, or keep a fixed rotation in the world
// - Generate random rotation angles like fxgrass does
// - Optionally Sort items based on depth
// - Optionally turn off depth buffer writes
// (use with 4 to get high quality rendering as long as your items don't overlap eachother
// , usefull for certain small batches of foliage)
// - Optionally fade out as you start to view an item edge on
// so you don't see the hard lines.
// - Batch rendering of foliage items into 1024 chunks.
// Currently this is just prep work for getting them into a
// proper vertex buffer type of object.
// - Added support for height range, min and max allowable height
// - Added density limitation like fxGrass does,
// needs more work still but ok for now.
// - Changed fxFoliages render sort type to BeginSort,
// before it was PointSort which made it difficult to
// combine two fxFoliage items at one place
// - Got rid of some of the rendering settings that we weren't changing
// - aligned foliage to normal of point they are placed at
// - removed MatrixF from foliage item and used Point3F to conserve memory
//TODO: -clint
// - configurable rotation / leaning angle
// - option to have foliage face straight up, for things like clover fields
// and lily pads
// - finish with render batching support get the vertex data into agp memory
// - optional bottom luminance scale to simulate shadowing in thick
// patches of grass
// - cache foliage items so they don't have to be replicated each time
// - support 'relative slope' restrictions, so you can detect when you
// are in a ravine, or other protected spot where plants are more likely
// to grow
// - in the end, I would like to see this be owned by the terrain more
// and closely tied to the terrain materials with replication and
// foliage lod done on an as needed basis
// - see if we can use DistanceSquared instead of Distance when rendering
// our foliage to avoid the sqrt
//-----------------------------------------------------------------------------
//------------------------------------------------------------------------------
//
// Put this in /example/common/client/missionDownload.cs in [function sceneLightingComplete()] (approx line 92)
// after codeline 'onPhase3Complete();'.
//
// echo("*** FXFoliage replication started");
// StartFoliageReplication();
//
// Note that this used to be in a different place, but now that fxFoliage
// uses the terrain lightmap to determine it's lighting it had to be moved
// to this point after the scene lighting is finished
// -Clint
//------------------------------------------------------------------------------and here are a couple fxFoliage data blocks from my mission file I was using to test with:
new fxFoliageReplicator(grassFAR) {
position = "-28.3703 311.656 191.186";
rotation = "1 0 0 0";
scale = "1 1 1";
UseDebugInfo = "0";
DebugBoxHeight = "1";
HideFoliage = "0";
ShowPlacementArea = "0";
PlacementAreaHeight = "25";
PlacementColour = "0.400000 0.000000 0.800000 1.000000";
Seed = "1376312589";
FoliageFile = "~/data/shapes/plants/grass_C.png";
FoliageCount = "50000";
FoliageRetries = "10";
InnerRadiusX = "10";
InnerRadiusY = "10";
OuterRadiusX = "512";
OuterRadiusY = "512";
MinWidth = "3";
MaxWidth = "3";
MinHeight = "1";
MaxHeight = "1";
FixAspectRatio = "0";
FixSizeToMax = "0";
OffsetZ = "0";
RandomFlip = "1";
UseCulling = "1";
CullResolution = "16";
ViewDistance = "150";
ViewClosest = "30";
FadeInRegion = "50";
FadeOutRegion = "5";
AlphaCutoff = "0.1";
GroundAlpha = "0.5";
ShrinkOnFade = "1";
DistanceSortOn = "0";
DepthWrite = "1";
FadeEdgeOn = "0";
AxisAligned = "0";
SwayOn = "1";
SwaySync = "0";
SwayMagSide = "0.1";
SwayMagFront = "0.2";
MinSwayTime = "5";
MaxSwayTime = "10";
LightOn = "1";
LightSync = "0";
MinLuminance = "0.7";
MaxLuminance = "1";
LightTime = "5";
AllowOnTerrain = "1";
AllowOnInteriors = "0";
AllowOnStatics = "0";
AllowOnWater = "0";
AllowWaterSurface = "0";
AllowedTerrainSlope = "30";
MinAllowedTerrainHeight = "-20000";
MaxAllowedTerrainHeight = "200000";
};
new fxFoliageReplicator(grassNear) {
position = "-28.3703 311.656 191.186";
rotation = "1 0 0 0";
scale = "1 1 1";
UseDebugInfo = "0";
DebugBoxHeight = "1";
HideFoliage = "0";
ShowPlacementArea = "0";
PlacementAreaHeight = "25";
PlacementColour = "0.400000 0.000000 0.800000 1.000000";
Seed = "1376312589";
FoliageFile = "~/data/shapes/plants/grass_C.png";
FoliageCount = "600000";
FoliageRetries = "10";
InnerRadiusX = "10";
InnerRadiusY = "10";
OuterRadiusX = "512";
OuterRadiusY = "512";
MinWidth = "4";
MaxWidth = "6";
MinHeight = "1";
MaxHeight = "1";
FixAspectRatio = "0";
FixSizeToMax = "0";
OffsetZ = "0";
RandomFlip = "1";
UseCulling = "1";
CullResolution = "8";
ViewDistance = "25";
ViewClosest = "1";
FadeInRegion = "10";
FadeOutRegion = "10";
AlphaCutoff = "0.1";
GroundAlpha = "0.5";
ShrinkOnFade = "1";
DistanceSortOn = "0";
DepthWrite = "1";
FadeEdgeOn = "0";
AxisAligned = "0";
SwayOn = "1";
SwaySync = "0";
SwayMagSide = "0.1";
SwayMagFront = "0.2";
MinSwayTime = "5";
MaxSwayTime = "10";
LightOn = "1";
LightSync = "0";
MinLuminance = "0.7";
MaxLuminance = "1";
LightTime = "5";
AllowOnTerrain = "1";
AllowOnInteriors = "0";
AllowOnStatics = "0";
AllowOnWater = "0";
AllowWaterSurface = "0";
AllowedTerrainSlope = "30";
MinAllowedTerrainHeight = "-20000";
MaxAllowedTerrainHeight = "200000";
};
new fxFoliageReplicator(plants) {
position = "-40.949 265.848 174.446";
rotation = "1 0 0 0";
scale = "1 1 1";
UseDebugInfo = "0";
DebugBoxHeight = "1";
HideFoliage = "0";
ShowPlacementArea = "0";
PlacementAreaHeight = "25";
PlacementColour = "0.400000 0.000000 0.800000 1.000000";
Seed = "2050677939";
FoliageFile = "~/data/uegmade/shapes/plants/plant3.png";
FoliageCount = "10000";
FoliageRetries = "10";
InnerRadiusX = "10";
InnerRadiusY = "10";
OuterRadiusX = "256";
OuterRadiusY = "256";
MinWidth = "0.5";
MaxWidth = "0.5";
MinHeight = "1";
MaxHeight = "2";
FixAspectRatio = "0";
FixSizeToMax = "0";
OffsetZ = "0";
RandomFlip = "1";
UseCulling = "1";
CullResolution = "8";
ViewDistance = "35";
ViewClosest = "1";
FadeInRegion = "10";
FadeOutRegion = "10";
AlphaCutoff = "0.1";
GroundAlpha = "0.5";
ShrinkOnFade = "0";
DistanceSortOn = "0";
DepthWrite = "1";
FadeEdgeOn = "0";
AxisAligned = "1";
SwayOn = "1";
SwaySync = "0";
SwayMagSide = "0.1";
SwayMagFront = "0.2";
MinSwayTime = "5";
MaxSwayTime = "10";
LightOn = "1";
LightSync = "0";
MinLuminance = "0.7";
MaxLuminance = "1";
LightTime = "5";
AllowOnTerrain = "1";
AllowOnInteriors = "0";
AllowOnStatics = "0";
AllowOnWater = "0";
AllowWaterSurface = "0";
AllowedTerrainSlope = "10";
MinAllowedTerrainHeight = "-20000";
MaxAllowedTerrainHeight = "200000";
};edit 01/11/2005
- Added support for cacheing foliage to .mfr files. Similar to .ml lighting cache files.
It's turned off by default, but you can set
$pref::foliage::cacheFoliage =1;
to activate it. It's turned off by default because the files it generates can be large( even with zip compression ) and there is no support for automatically deleting old or invalid files.
- Used a DataChunker to allocate the foliage items, should result in better performance as foliage item memory is allocated together.
- Cleaned up the foliage console output, added a nice summary and printout of total foliage replication time.
I had hoped to save a lot of foliage generation time and at first I had, but in looking at things more closely it seems that foliage generation time wasn't that bad in the first place, it's other object creation that seems to be slowing my own mission down. have to look into that.
When using cacheing the first time, foliage generation is a lot slower since it has to write about a 30 meg compressed file for one of my scenes, but when loading a cached file it only takes about half the time of the original. So it's a real tradeoff in space vs loading time, hence the default to not caching.
After using the DataChunker to allocate memory everything felt faster, but actual benchmarking results were inconclusive you try and let me know :)
#2
12/28/2004 (3:30 pm)
Can't download, the link points to this page.
#3
12/28/2004 (3:37 pm)
wierd, ok re-uploaded zip file.
#6
should be something more like
to make sure the transformed boundingbox covers at least as much space as the original. With the original code, the bounding boxes representing the quadtree nodes may be incorrect if the foliage scene object is rotated.
It seems to have fixed my "vanishing chunk" problems but I'll need to test it out a bit more. I haven't checked the original foliage or shape replicators.
Edit: Well, apparently there was one guy talking about the grass "vanishing at certain angles" but that problem appears to have been fixed AFAICT from the foroms. The effect I'm seeing probably does not show up too often since people usually do not rotate their foliage objects.
12/29/2004 (9:33 am)
Clint, I seem to have whole quadtree chunks of foliage vanishing at certain positions and angles. I dunno, but it seems like this code at the start of fxFoliageRenderList::IsQuadrantVisible:InvXForm.mulP(OSBox.min); InvXForm.mulP(OSBox.max);
should be something more like
InvXForm.mul(OSBox);
to make sure the transformed boundingbox covers at least as much space as the original. With the original code, the bounding boxes representing the quadtree nodes may be incorrect if the foliage scene object is rotated.
It seems to have fixed my "vanishing chunk" problems but I'll need to test it out a bit more. I haven't checked the original foliage or shape replicators.
Edit: Well, apparently there was one guy talking about the grass "vanishing at certain angles" but that problem appears to have been fixed AFAICT from the foroms. The effect I'm seeing probably does not show up too often since people usually do not rotate their foliage objects.
#7
I think that first chunk of code should be ok though
it might be that I've messed up the transform that gets passed into that function, I'll try it out.
12/29/2004 (2:24 pm)
Hi Gary, I haven't really tested with rotated or non circular foliage so there could definitely be a problem.I think that first chunk of code should be ok though
it might be that I've messed up the transform that gets passed into that function, I'll try it out.
#8
how so? isn't the inverse of rotation and translation valid?
ahhhhh it's an axis aligned box, ok I see I didn't realize that, yes then what you did sounds like the way to go I'll change it...
12/29/2004 (2:43 pm)
ok I see the problem too with a rotated foliageQuote:o make sure the transformed boundingbox covers at least as much space as the original. With the original code, the bounding boxes representing the quadtree nodes may be incorrect if the foliage scene object is rotated.
how so? isn't the inverse of rotation and translation valid?
ahhhhh it's an axis aligned box, ok I see I didn't realize that, yes then what you did sounds like the way to go I'll change it...
#10
12/30/2004 (10:52 am)
Clint, I forgot to mention that the new foliage is super-cool, especially with the lighting and random rotations. I'm dumping grass pretty much everywhere in my mission now...
#11
I think the next high priority thing to do here is to cache the replication so it doesn't have to be done each time. Everytime I start my mission to test some code I can't help but think of the lost seconds :)
we'll have to look at how the lighting handles caching and copy it.
12/30/2004 (3:19 pm)
glad to help!I think the next high priority thing to do here is to cache the replication so it doesn't have to be done each time. Everytime I start my mission to test some code I can't help but think of the lost seconds :)
we'll have to look at how the lighting handles caching and copy it.
#12
12/30/2004 (8:57 pm)
Great resource. I couldn't get it to work with the TWSurfaceReference object, so I had to revert back to my old FX foliage. I loved the lighting so I ripped it out and have it working in the foliage and grass replicators.
#13
I thought about putting TWSurfaceReference in like fxgrass had, but I wanted it to work easily with the default TGE 1.3
I haven't tried the surface reference stuff, I'll try it out whenever I get a chance, and see if I can integrate it here easily also.
01/01/2005 (4:28 pm)
@Josh, do you think that was because of some of the changes I made?I thought about putting TWSurfaceReference in like fxgrass had, but I wanted it to work easily with the default TGE 1.3
I haven't tried the surface reference stuff, I'll try it out whenever I get a chance, and see if I can integrate it here easily also.
#14
01/01/2005 (6:07 pm)
Must be. How'd you get the grass looking so good with this class? I tried making some grass with it, but it looked really crappy. Pobably just my mapping skills, lol.
#15
I just used the datablocks I posted. One difference is that the ordering of fxFoliage in your mission file will make a difference in the rendering. The only real difference between this and fxGrass that I could tell in the code is that fxGrass will render two polys for every foliage item in an x split pattern unless I read it wrong. oh and this doesn't color the grass for you so you have to use a grass color texture.
01/01/2005 (6:56 pm)
heh, it's a tiny screenshot if it were big it might look crappy too :) I just used the datablocks I posted. One difference is that the ordering of fxFoliage in your mission file will make a difference in the rendering. The only real difference between this and fxGrass that I could tell in the code is that fxGrass will render two polys for every foliage item in an x split pattern unless I read it wrong. oh and this doesn't color the grass for you so you have to use a grass color texture.
#16
01/01/2005 (7:42 pm)
Oh ya, most of my grass textures don't have color. So I was using a really bad one that did, which made it not look so good. Great work on the FX foliage, BTW. You should try fixing up the FX grass. I hear that the rendering could really be improved upon(hint, hint....). ;)
#17
Thanks.
01/04/2005 (6:22 am)
Thanks for providing this, I can't wait to see how it works. One question, I downloaded the resource but there is no grass texture file. Does anybody have a nice one I can use?Thanks.
#18
i couldnt get it to work, i thought it was the resource.
i loaded it into the demo and it worked great!! really fast too
for some reason culling isnt working on my mission though.. i must be skipping something it needs i dont know..
any idea why it wouldnt work?
01/06/2005 (7:20 am)
i dont know why culling isnt working on my level..i couldnt get it to work, i thought it was the resource.
i loaded it into the demo and it worked great!! really fast too
for some reason culling isnt working on my mission though.. i must be skipping something it needs i dont know..
any idea why it wouldnt work?
#19
@Charles: glad you got it working in the demo. if your cull resolution is smaller than your width of the foliage items then it doesn't work correctly. At least that's what I've seen. If you post your datablock from your .mis file for the foliage perhaps we can tell what settings are wrong.
01/06/2005 (12:39 pm)
@joe: I just used the texture that came with the fxgrass resource, but yuo have to color it the color you want. this (like fxfoliage) doesn't use vertex colors so the color comes from your mission lighting and from your texture map.@Charles: glad you got it working in the demo. if your cull resolution is smaller than your width of the foliage items then it doesn't work correctly. At least that's what I've seen. If you post your datablock from your .mis file for the foliage perhaps we can tell what settings are wrong.
#20
the only problem is that the cached files can get very big. with my test mission one of the foliage files that has tons of foliageitems ran up to 58 megs, I changed the caching to use a zip stream instead and now it's only about 30 megs which is nicer. replication time went from about 16 seconds when not cached to about 3 seconds when cached. I'm sure that could be improved on as well, but that's good enough for me right now :)
will update soon.
01/10/2005 (7:48 pm)
almost done with cacheing foliage replication ...need to add some preferences to do it or not do it and do some more testing then I'll update this resource.the only problem is that the cached files can get very big. with my test mission one of the foliage files that has tons of foliageitems ran up to 58 megs, I changed the caching to use a zip stream instead and now it's only about 30 megs which is nicer. replication time went from about 16 seconds when not cached to about 3 seconds when cached. I'm sure that could be improved on as well, but that's good enough for me right now :)
will update soon.
Associate Ben Garney