It's Teh Economy Stoopid
by Steve Acaster · 04/06/2009 (3:15 pm) · 31 comments
Actually, it's the lightmaps. Constructor crashes on export? Bets are it's the lightmaps. Funny dark cuts along the edges of the brushes visible from certain angles and certain distances? Not brush borders, but lightmap borders.
Why does this all happen? Because geometry scale is 32, small scale lightmaps (like 8) cause trauma on larger brushes when calculating the lightmap itself. Keep brushes smaller for less lightmap calculation map crash trauma. Compressed edges of lightmaps visible? Make lightmap scale the same as geometry to dispell (though to be honest you'll probably still see a few of the little critters lurking out in the far distance if you have a long, long surface like a road). There's also an occaisonal light bleed in some interiors with this.
What the world looks like immediately after it spawns from my id
So, with that finally realised, we set out to design Map2. Map1 (aka High Wold, it's the urban pics in previous blogs) was created as a large test DIF using large facade blocks as the buildings. Rather urbanized, terrace streets. Map2 was designed for more individual buildings, detached, semi-detached housing. The idea being to create a series of prefabs which I can then drop into editor/modeling program, arrange and then export en mass as a single object (DIF or DTS).
And it all seemed to go okay like that. I did attempt a new method of creating a mesh model in Blender and then exporting it as a mapfile, but Blender's map exporter isn't up to anything particularly complicated, and the end result looked like a mangled, disjointed mess on close inspection. Blender can export mapfiles fine - if you stick to the basic rules of being BSP friendly (build it using individual blocks - think lego).
Church tower is 400m away, just out of rifle range but not sniper or lmg
So, establishing that prefabbing really is the way to go, and with a DIF version knocked up, I recreated the whole thing as a DTS. Whilst polysoup collision has been available for a while, the lack of lightmaps has meant that it just didn't cast realistic shadows onto itself or other meshes. This can look a bit odd to say the least. However, there is new tech on the horizon which may well sort this problem out, and which includes scandulous mention (it's not really scandulous, I'm just using a random adjective for teh interwebz draaama) that DIF will be more performance friendly. With my Map2 copied into both DIF and Polysouped DTS, it'll be interesting to see what the difference is in-game using said new tech.
BSP version, roofs are NULLed
To be honest I was having so much fun building the DTS version that I imbellished it quite a bit, so it's got 7k faces instead of 5k as the DIF version. Performance-wise in current tech (TGEA1.7.1 'cos I haven't transfered all of my custom scripts over to 1.8.1 yet - though from what I've experimented with - the High Wold map, 1.8.1 gives noticeably better performance than 1.7.1) the DTS polysoup wins out with an FPS of 30% higher than DIF (based on 5 locations and views at 1280x768 resolution with 1700m visibility). That's an max-average fps of DIF 125 and DTS Polysoup 165.
Polysouped DTS version - spot the difference!
Apart from the DTS lighting issue there are also a couple of other problems, such as the model vanishing when the fog/maxvis hits the centre of the bounding box (problematic with large meshes in low visibility environments - but fine with a long maxvis like I'm currently using). I'm a great believer that in making large environments as a single object is better for performance than making hundreds of individual objects and then sticking them in-world together. No matter how many surfaces it has, it is still just one object for the processor to initialize and then worry about, instead of 50. Another issue I've noticed with current tech's polysoup is that the colour/tone/intensity of the textures aren't as vibrant as they are when applied to DIF. I expect this is down to the lightmaps.
Anyhow, it'll be interesting to see what the new tech brings. Roll on Beta.
Why does this all happen? Because geometry scale is 32, small scale lightmaps (like 8) cause trauma on larger brushes when calculating the lightmap itself. Keep brushes smaller for less lightmap calculation map crash trauma. Compressed edges of lightmaps visible? Make lightmap scale the same as geometry to dispell (though to be honest you'll probably still see a few of the little critters lurking out in the far distance if you have a long, long surface like a road). There's also an occaisonal light bleed in some interiors with this.
What the world looks like immediately after it spawns from my id
So, with that finally realised, we set out to design Map2. Map1 (aka High Wold, it's the urban pics in previous blogs) was created as a large test DIF using large facade blocks as the buildings. Rather urbanized, terrace streets. Map2 was designed for more individual buildings, detached, semi-detached housing. The idea being to create a series of prefabs which I can then drop into editor/modeling program, arrange and then export en mass as a single object (DIF or DTS).
And it all seemed to go okay like that. I did attempt a new method of creating a mesh model in Blender and then exporting it as a mapfile, but Blender's map exporter isn't up to anything particularly complicated, and the end result looked like a mangled, disjointed mess on close inspection. Blender can export mapfiles fine - if you stick to the basic rules of being BSP friendly (build it using individual blocks - think lego).
Church tower is 400m away, just out of rifle range but not sniper or lmg
So, establishing that prefabbing really is the way to go, and with a DIF version knocked up, I recreated the whole thing as a DTS. Whilst polysoup collision has been available for a while, the lack of lightmaps has meant that it just didn't cast realistic shadows onto itself or other meshes. This can look a bit odd to say the least. However, there is new tech on the horizon which may well sort this problem out, and which includes scandulous mention (it's not really scandulous, I'm just using a random adjective for teh interwebz draaama) that DIF will be more performance friendly. With my Map2 copied into both DIF and Polysouped DTS, it'll be interesting to see what the difference is in-game using said new tech.
BSP version, roofs are NULLed
To be honest I was having so much fun building the DTS version that I imbellished it quite a bit, so it's got 7k faces instead of 5k as the DIF version. Performance-wise in current tech (TGEA1.7.1 'cos I haven't transfered all of my custom scripts over to 1.8.1 yet - though from what I've experimented with - the High Wold map, 1.8.1 gives noticeably better performance than 1.7.1) the DTS polysoup wins out with an FPS of 30% higher than DIF (based on 5 locations and views at 1280x768 resolution with 1700m visibility). That's an max-average fps of DIF 125 and DTS Polysoup 165.
Polysouped DTS version - spot the difference!
Apart from the DTS lighting issue there are also a couple of other problems, such as the model vanishing when the fog/maxvis hits the centre of the bounding box (problematic with large meshes in low visibility environments - but fine with a long maxvis like I'm currently using). I'm a great believer that in making large environments as a single object is better for performance than making hundreds of individual objects and then sticking them in-world together. No matter how many surfaces it has, it is still just one object for the processor to initialize and then worry about, instead of 50. Another issue I've noticed with current tech's polysoup is that the colour/tone/intensity of the textures aren't as vibrant as they are when applied to DIF. I expect this is down to the lightmaps.
Anyhow, it'll be interesting to see what the new tech brings. Roll on Beta.
About the author
One Bloke ... In His Bedroom ... Trying To Make Computer Games ... eventually ...
#22
04/07/2009 (2:26 pm)
So, the lightmaps are plotting to kill us?
#23
Kent Brockman: "Professor, without knowing precisely what the danger is, would you say it's time for our viewers to crack each other's heads open and feast on the goo inside?"
Professor: "Yes I would, Kent."
edit: Whoa! Website update!
04/07/2009 (2:42 pm)
@SteveKent Brockman: "Professor, without knowing precisely what the danger is, would you say it's time for our viewers to crack each other's heads open and feast on the goo inside?"
Professor: "Yes I would, Kent."
edit: Whoa! Website update!
#24
Except in this case, optimum performance would be realized via per object LOD. With one ginormous object, you cannot use LOD; you're always "inside" the object, never far enough away for LOD to actually work.
Additionally, by splitting up the world into separate objects, you can completely avoid the issue of your town vanishing if the model's origin passes the fog boundary.
04/09/2009 (4:13 pm)
Quote:"...I'm a great believer that in making large environments as a single object is better for performance than making hundreds of individual objects and then sticking them in-world together. No matter how many surfaces it has, it is still just one object for the processor to initialize and then worry about, instead of 50..."
Except in this case, optimum performance would be realized via per object LOD. With one ginormous object, you cannot use LOD; you're always "inside" the object, never far enough away for LOD to actually work.
Additionally, by splitting up the world into separate objects, you can completely avoid the issue of your town vanishing if the model's origin passes the fog boundary.
#25
I tried the LOD thing first, but the performance wasn't so great because of the engine instancing of so many individual objects. Doing it all "en mass" this way gave better performance, whether DTS or DIF. There's also a slight issue of polysoup collision objects causing very bad performance if they overlap a little, and aligning things up is a real pain.
Small details such as street signs, objects, et al are individual objects and are thus LODed accordingly. It's more the "urban landscape" that is one huge model.
DIF doesn't suffer from the "vanishing" problem that mesh models do, and actually culls quite nicely (if you switch view interiors to zones or outline you can see it cull through the fog).
04/09/2009 (7:35 pm)
@ KevinI tried the LOD thing first, but the performance wasn't so great because of the engine instancing of so many individual objects. Doing it all "en mass" this way gave better performance, whether DTS or DIF. There's also a slight issue of polysoup collision objects causing very bad performance if they overlap a little, and aligning things up is a real pain.
Small details such as street signs, objects, et al are individual objects and are thus LODed accordingly. It's more the "urban landscape" that is one huge model.
DIF doesn't suffer from the "vanishing" problem that mesh models do, and actually culls quite nicely (if you switch view interiors to zones or outline you can see it cull through the fog).
#26
Whilst tested with the DTS version it produced 210fps in x768 and 155fps in x1050.
04/10/2009 (2:55 pm)
It should be noted that as 90% of the buildings are facades, the textures are 1024x. There are 5 wall types, each with 3 window/door types, so that's 15 1024x facade textures. The wall/brick, window and doors on these textures are replicated elsewhere in the map (walls at 512x, doors, windows at 256x). Dropping the 1024x facade textures in favour of more complex geometry in the buildings (eg: modeling the window and doors individually with the smaller textures) gives a considerable improvement in performance.Whilst tested with the DTS version it produced 210fps in x768 and 155fps in x1050.
#27
What I had done was to make my own trials on a crappy laptop of mine -- the lowest common denominator of what the "average" computer user may be using. 2 GHz Dual Core Athlon, 3 GB of Ram, and nVidia graphics with 64 MB of VideoRAM, at 1280 X 800 resolution, running Windows Vista with Aero turned on. I used this setup since I was sure it would give me low results.
I had a collection of neo-Victorian style townhouses arranged in a two block area along with a few warehouses in a "dock" area, 14 difs in all,
Before: 16 fps -- 14 difs no LOD
Before: 21 fps -- 14 difs with LOD
After: 25 fps -- 1 merged dif
I haven't tried the same experiment with dts's yet, but I think the results speak rather well for the performance increase of your approach while using difs. Next up, since this laptop only has 64 MB of VRAM I think I might optimize the textures... since some of them are so large: 1024 even a few 2048).
04/10/2009 (5:10 pm)
About 2 hours ago silly me had a nice long post about how your "en masse" approach made sense to me so I tried it for myself, but forgot that the internet doesn't work when the power is out ;)What I had done was to make my own trials on a crappy laptop of mine -- the lowest common denominator of what the "average" computer user may be using. 2 GHz Dual Core Athlon, 3 GB of Ram, and nVidia graphics with 64 MB of VideoRAM, at 1280 X 800 resolution, running Windows Vista with Aero turned on. I used this setup since I was sure it would give me low results.
I had a collection of neo-Victorian style townhouses arranged in a two block area along with a few warehouses in a "dock" area, 14 difs in all,
Before: 16 fps -- 14 difs no LOD
Before: 21 fps -- 14 difs with LOD
After: 25 fps -- 1 merged dif
I haven't tried the same experiment with dts's yet, but I think the results speak rather well for the performance increase of your approach while using difs. Next up, since this laptop only has 64 MB of VRAM I think I might optimize the textures... since some of them are so large: 1024 even a few 2048).
#28
But yeah, I noticed Matt Fairfax mention something about max resolution of textures for T3D today and that reminded me of my large (1024) facade textures and the Vram they use, so as a quick experiment I swapped them for 512s used elsewhere.
I'm going to redesign my Level2 DTS town, and make all of the buildings fully enterable, with stairs, open windows, the whole lot polysouped en mass just to see what the performance is like then.
I have found one problem with polysouped DTS in TGEA1.7.1 though, whilst mucking about bouncing grenades around corners in a house, I discovered that the blast radius of weapons goes through the walls even if the actual projectile doesn't.
04/10/2009 (6:45 pm)
Quote:some of them are so large: 1024 even a few 20482048!? I feel faint at the very thought!
But yeah, I noticed Matt Fairfax mention something about max resolution of textures for T3D today and that reminded me of my large (1024) facade textures and the Vram they use, so as a quick experiment I swapped them for 512s used elsewhere.
I'm going to redesign my Level2 DTS town, and make all of the buildings fully enterable, with stairs, open windows, the whole lot polysouped en mass just to see what the performance is like then.
I have found one problem with polysouped DTS in TGEA1.7.1 though, whilst mucking about bouncing grenades around corners in a house, I discovered that the blast radius of weapons goes through the walls even if the actual projectile doesn't.
#29
04/10/2009 (7:39 pm)
Quote:I have found one problem with polysouped DTS in TGEA1.7.1 though, whilst mucking about bouncing grenades around corners in a house, I discovered that the blast radius of weapons goes through the walls even if the actual projectile doesn't.Just a thought, but have you tried adding StaticShapes (or is it TSStatics) to your TypeMasks when you check for explosion coverage (calcExplosionCoverage) in your radiusDamage function? I think by default it only checks against Interiors, Terrain, and Vehicles to see if the explosion is blocked. If you do that though I would wonder what happens to someone standing behind a lamppost hiding from a grenade....
#30
First off, I'm using TGE 1.5.2. Second, my interiors are significantly more detailed than your. My interiors aren't just facades; you can actually go inside them. Because of that, my interiors have multiple times as many brushes as they otherwise would.
Since some of my interiors are so detailed, I've actually cut up some of the larger ones into smaller pieces. This lets one half of a building switch to a lower LoD while the player is inside the other half.
In any case, these are still some interesting ideas and results.
04/10/2009 (9:38 pm)
After reading about your results I decided to try making all of my interiors as one large object instead of multiple smaller objects that have LoDs. And my performance actually got worse (from around 25 fps to 20 on low-ish end hardware), but this was likely because my situation is a bit different from the one you seen to be testing.First off, I'm using TGE 1.5.2. Second, my interiors are significantly more detailed than your. My interiors aren't just facades; you can actually go inside them. Because of that, my interiors have multiple times as many brushes as they otherwise would.
Since some of my interiors are so detailed, I've actually cut up some of the larger ones into smaller pieces. This lets one half of a building switch to a lower LoD while the player is inside the other half.
In any case, these are still some interesting ideas and results.
#31
Yeah, I upgraded to TGEA mainly for the performance boost my stuff got over TGE. If you're doing very detailed stuff, then portalling is probably essential, though for my style of large open environment it doesn't work so well. Back in the day of modding maps for CoD I could just portal every window and door and let the engine deal with it.
@Michael
Hadn't occured to me to check if it was something missing from typemasks.
04/11/2009 (6:30 am)
@TwistedYeah, I upgraded to TGEA mainly for the performance boost my stuff got over TGE. If you're doing very detailed stuff, then portalling is probably essential, though for my style of large open environment it doesn't work so well. Back in the day of modding maps for CoD I could just portal every window and door and let the engine deal with it.
@Michael
Hadn't occured to me to check if it was something missing from typemasks.
Quote:I would wonder what happens to someone standing behind a lamppost hiding from a grenade....Probably depends how wide the lamppost is! I'll look at the code and see if it uses any special part of the entity bounding box (like the origin) to calculate if the entity is hit or not.
Torque 3D Owner Andy Wright
BrokeAss Games