DIF and engine performance
by Tom Petrus · in Artist Corner · 02/28/2006 (7:56 am) · 22 replies
I have a few questions on what the "best practices" would be for creating DIF files for the Torque engine.
I am assuming, as with any other 3d file type, you would want to keep the number of faces as low as possible while still having a visually acceptable DIF.
-> In a situation where you could use several bushes or a more complicated brush, does it make a difference in the way it compiles or with game performance?
For instance: You would like to split a 'wall' into 3 sections for texturing, If you use 3 cube primitives to make the wall you'd have your 3 sections. There would be 4 faces that would be connected and hidden. -> not used in comipile?
________ ________ _______
[_______][________][_______]
You could also use a cylinder primitive with the proper number of sides and vertex edit that into a 3 section wall.
______.________.______
[_____.________.______]
Or you may even be able to use 1 cube primitive and create a large single texture with the art for all 3 wall sections. (Provided you can make it look good stretching the texture.)(I didn't like this idea but it was sugested so I'm tossing it out there.)
Does the method make a difference when the DIF is compiled or when it is read and displayed by the engine?
What's best on performance? Multiple simple brushes, 1 complicated brush, Single simple brush with huge texture when possible?
-> Un-seen faces: If you have a face on a DIF that will never be seen, can you remove that face to save the engine some work. (I have tried but haven't been able to get the file to export.)
For Instance: The bottom of a strutcture will always be under or equal to the terain height and never be seen ingame.
Is there a way to remove that face and compile a DIF. Would it make a difference in the engine if you can?
-> LoD - Level of Detail: When making multiple levels of detail for a DIF does it make sense to change the textures as well as the geometry?
For Instance: You use a 512x512 detailed texture when you are up close - when you are too far away and the engine changes DIF LoD the detail on the texture isn't seen. (Too far away - DIF is tiny on screen.) It would probably look fine with a 128x128 texture.
Does it effect rendering performance to change the texture to a smaller resource or since the 512 texture is already in memory does it make more since to just leave it and save some memory.
-> Detail Brushes: Is a detail brush more complicated from the engines point of view. Should detail brushes be used only when absolutley needed? How is performance effected with a detail brush - if any?
-> Object thickness...: This may sound wierd but I read somewhere (and I forget what engine was being discussed) that thicker walls were easier on engine performance. No explanation and it didn't make sense to me... Any info on that statement
?
--------------
I hope the post made sense. I'm trying to make 'smart' decisions in the construction phase so hopefully I don't need an overly complicated re-construction phase. hehehe
I am assuming, as with any other 3d file type, you would want to keep the number of faces as low as possible while still having a visually acceptable DIF.
-> In a situation where you could use several bushes or a more complicated brush, does it make a difference in the way it compiles or with game performance?
For instance: You would like to split a 'wall' into 3 sections for texturing, If you use 3 cube primitives to make the wall you'd have your 3 sections. There would be 4 faces that would be connected and hidden. -> not used in comipile?
________ ________ _______
[_______][________][_______]
You could also use a cylinder primitive with the proper number of sides and vertex edit that into a 3 section wall.
______.________.______
[_____.________.______]
Or you may even be able to use 1 cube primitive and create a large single texture with the art for all 3 wall sections. (Provided you can make it look good stretching the texture.)(I didn't like this idea but it was sugested so I'm tossing it out there.)
Does the method make a difference when the DIF is compiled or when it is read and displayed by the engine?
What's best on performance? Multiple simple brushes, 1 complicated brush, Single simple brush with huge texture when possible?
-> Un-seen faces: If you have a face on a DIF that will never be seen, can you remove that face to save the engine some work. (I have tried but haven't been able to get the file to export.)
For Instance: The bottom of a strutcture will always be under or equal to the terain height and never be seen ingame.
Is there a way to remove that face and compile a DIF. Would it make a difference in the engine if you can?
-> LoD - Level of Detail: When making multiple levels of detail for a DIF does it make sense to change the textures as well as the geometry?
For Instance: You use a 512x512 detailed texture when you are up close - when you are too far away and the engine changes DIF LoD the detail on the texture isn't seen. (Too far away - DIF is tiny on screen.) It would probably look fine with a 128x128 texture.
Does it effect rendering performance to change the texture to a smaller resource or since the 512 texture is already in memory does it make more since to just leave it and save some memory.
-> Detail Brushes: Is a detail brush more complicated from the engines point of view. Should detail brushes be used only when absolutley needed? How is performance effected with a detail brush - if any?
-> Object thickness...: This may sound wierd but I read somewhere (and I forget what engine was being discussed) that thicker walls were easier on engine performance. No explanation and it didn't make sense to me... Any info on that statement
?
--------------
I hope the post made sense. I'm trying to make 'smart' decisions in the construction phase so hopefully I don't need an overly complicated re-construction phase. hehehe
About the author
#2
I realise that its still converted to triangles, but are there any optimizations for rectangular shapes. Or is it simply faster to have as few triangles as possible, so would wedge and pyramid shaped brushes render faster than a cube.
And with Null faces, I realise that the triangles aren't rendered, but are they also dropped out of the lighting as well as rendering phase and only used for collisions?
Just wondering what kind of optimization is done to null polys, and what processes are dropped other than the render pipeline?.
02/28/2006 (9:32 am)
Actually I have been wondering about some of these things too. I'm not 100% sure on how DIF is rendered. I realise that its still converted to triangles, but are there any optimizations for rectangular shapes. Or is it simply faster to have as few triangles as possible, so would wedge and pyramid shaped brushes render faster than a cube.
And with Null faces, I realise that the triangles aren't rendered, but are they also dropped out of the lighting as well as rendering phase and only used for collisions?
Just wondering what kind of optimization is done to null polys, and what processes are dropped other than the render pipeline?.
#3
I'm still experimenting to see how the rectangles are turned to triangles. Attempting to see if I can predict the splitting while I am designing. I have seen some wierd light shading on some faces and have broken the model into parts to try and dictacte where the triangles are created. Detail brushes are suppose to not fracture the surfaces they touch but I haven't figured out how to define a brush as a detail brush in Cartography Shop... Any ideas?
02/28/2006 (10:01 am)
Thanks for the answers and some good follow up questions.I'm still experimenting to see how the rectangles are turned to triangles. Attempting to see if I can predict the splitting while I am designing. I have seen some wierd light shading on some faces and have broken the model into parts to try and dictacte where the triangles are created. Detail brushes are suppose to not fracture the surfaces they touch but I haven't figured out how to define a brush as a detail brush in Cartography Shop... Any ideas?
#4
QuArK has there for spezial Torque Entities like:
- detail
- collision
- vehicle collision
- portal
under detail all brushes come which should be detailbrushes, under portal you can put the brushes which should be portals and collision and vehicle collision if you want to create an extra collision hull for your dif.
In the map file it shows up like following
So Cartography shop should support this and you have to luck for the detail entity, or if not you can save only the mapfile and modify the file with an Texteditor before you run the map2dif.exe ??
02/28/2006 (11:34 am)
I don't know anything about Cartography Shop but you have to put the brushes you want in to a detail entities group.QuArK has there for spezial Torque Entities like:
- detail
- collision
- vehicle collision
- portal
under detail all brushes come which should be detailbrushes, under portal you can put the brushes which should be portals and collision and vehicle collision if you want to create an extra collision hull for your dif.
In the map file it shows up like following
Quote:
// Entity 1
// detail:b[2]
{
"classname" "detail"
// Brush 0
// detail:b[2] -> Catwalk:g[1] -> cube:p[1]
{
( 106 0 0 ) ( 234 0 0 ) ( 106 128 0 ) NULL [ 1.00000 0.00000 0.00000 -212.00000 ] [ 0.00000 1.00000 0.00000 0.00000 ] 0 0.50000 -0.50000
( 106 0 8 ) ( 106 128 8 ) ( 234 0 8 ) NULL [ 1.00000 0.00000 0.00000 -212.00000 ] [ 0.00000 1.00000 0.00000 0.00000 ] 0 0.50000 -0.50000
( 106 -120 64 ) ( 106 -120 192 ) ( 234 -120 64 ) wood_3 [ 1.00000 0.00000 0.00000 -212.00000 ] [ 0.00000 0.00000 1.00000 128.00000 ] 0 0.50000 -0.50000
( 106 56 64 ) ( 234 56 64 ) ( 106 56 192 ) wood_3 [ 1.00000 0.00000 0.00000 -212.00000 ] [ 0.00000 0.00000 1.00000 128.00000 ] 0 0.50000 -0.50000
( 100 0 64 ) ( 100 128 64 ) ( 100 0 192 ) wood_3 [ 0.00000 1.00000 0.00000 0.00000 ] [ 0.00000 0.00000 1.00000 128.00000 ] 0 0.50000 -0.50000
( 108 0 64 ) ( 108 0 192 ) ( 108 128 64 ) wood_3 [ 0.00000 1.00000 0.00000 0.00000 ] [ 0.00000 0.00000 1.00000 128.00000 ] 0 0.50000 -0.50000
}
}
So Cartography shop should support this and you have to luck for the detail entity, or if not you can save only the mapfile and modify the file with an Texteditor before you run the map2dif.exe ??
#5
It seems the process is similiar in CShop - You group your detail brushes, select your group and add a classname key with the value set to detail.
02/28/2006 (12:35 pm)
Thanks for the info. It seems the process is similiar in CShop - You group your detail brushes, select your group and add a classname key with the value set to detail.
#6
The first and most talked about is that it makes the collision calculations much faster. On modern processors this is becoming less important but there is still definitely a loss of performance between a general polysoup collision scheme and a convex-only scheme.
The second reason is that it makes it trivial to do automatic hidden surface removal. In this scenario:
________ ________ ________
[_______][________][_______]
the surfaces that are butted up against each other are going to automatically get removed when the .map gets converted into a .dif. You don't need to texture them in any special way (NULL). You don't need to create complicated brushes (I recommend against it since it can cause all sorts of issues). You don't need to create a three-for-one texture (can hurt rendering performance...though it will help collision performace). Keep your brushes simple and your textures small and tight (and repeated as much as possible).
The third reason is that convex geometry makes it a lot easier and a lot faster to handle visibility and occlusion. It is easier to check a room made up of convex spaces for leaks. It is also easier on average for an artist to place a single portal brush across an opening than for them to have to hand create visibility groups.
As far as detail brushes go: detail brushes are excluded from the hidden surface removal pass so they can hurt rendering performance. You have to balance that against how much intersecting brushes can complicate the collision calculations by splitting up larger brushes into a bunch of smaller ones. How much and when to use them is more of an art than a science and you are going to have to play around a bit to see how it effects your games in particular.
02/28/2006 (12:36 pm)
There are three big reasons for the limitations on brush-based geometry (i.e. closed convex volumes).The first and most talked about is that it makes the collision calculations much faster. On modern processors this is becoming less important but there is still definitely a loss of performance between a general polysoup collision scheme and a convex-only scheme.
The second reason is that it makes it trivial to do automatic hidden surface removal. In this scenario:
________ ________ ________
[_______][________][_______]
the surfaces that are butted up against each other are going to automatically get removed when the .map gets converted into a .dif. You don't need to texture them in any special way (NULL). You don't need to create complicated brushes (I recommend against it since it can cause all sorts of issues). You don't need to create a three-for-one texture (can hurt rendering performance...though it will help collision performace). Keep your brushes simple and your textures small and tight (and repeated as much as possible).
The third reason is that convex geometry makes it a lot easier and a lot faster to handle visibility and occlusion. It is easier to check a room made up of convex spaces for leaks. It is also easier on average for an artist to place a single portal brush across an opening than for them to have to hand create visibility groups.
As far as detail brushes go: detail brushes are excluded from the hidden surface removal pass so they can hurt rendering performance. You have to balance that against how much intersecting brushes can complicate the collision calculations by splitting up larger brushes into a bunch of smaller ones. How much and when to use them is more of an art than a science and you are going to have to play around a bit to see how it effects your games in particular.
#7
Keep your textures small and tight. As small as possible while maintaining a good look... How large of a texture is too large? I have a few 512x512 textures that get placed on large surfaces. (Seems to work ok in the engine but my testing mission file doesn't have much on it.)
Are transparent textures supported? If not in a DIF maybe there is a way to use a transparant sprite in the engine? I am thinking of a grate, chain link fence, barbed wire, bunch of wires or another instance when the Object would be too complex to construct - too many faces.
03/01/2006 (6:49 am)
Thanks, Matt. If I understand correctly, it would be best to use simple brushes then a complex brush. The butted faces are removed on compile so no need to NULL texture them for removal from the engine. Detail brushes are a bigger hit on engine performance so use them wisely and test, test, test.Keep your textures small and tight. As small as possible while maintaining a good look... How large of a texture is too large? I have a few 512x512 textures that get placed on large surfaces. (Seems to work ok in the engine but my testing mission file doesn't have much on it.)
Are transparent textures supported? If not in a DIF maybe there is a way to use a transparant sprite in the engine? I am thinking of a grate, chain link fence, barbed wire, bunch of wires or another instance when the Object would be too complex to construct - too many faces.
#8
03/01/2006 (1:53 pm)
Transparency is supported on .dif's only in TSE. If you need transparency, you will need to use a .dts.
#9
03/01/2006 (1:56 pm)
Thanks, Matt!
#10
1) I have two DIF files loaded into Torque --- one simple and one complex. They are placed in the mission such that they are physically intersecting. I've noticed that when the simple DIF comes slightly into view (with complex DIF not in view), the performance behaves as badly as if the complex DIF were in view. So, I am wondering...how does the engine handle visibility and occlusion when two DIFs are intersecting? Why such a performance drop when I am only viewing the simple DIF?
2) Different scenario --- I have one complex DIF. It appears that even when I have a very simple piece of that DIF in view, the performance is not much better than if I were looking at the most complex part. I assumed the performance would be better, considering the whole point of using BSP models is to improve these sorts of visibility/occlusion issues. Why is the engine rendering things that are not in view? (And I am not just talking slightly out of view...but waaaaay out of view.)
I'm trying to put it all together here...When you are looking at a surface of a DIF, does the engine also render all the geometry that is connected to that surface? Or does it render the entire zone that the surface is a part of? What DOES it do?
Please help!
11/15/2006 (2:43 pm)
This is great thread! I have some further questions...1) I have two DIF files loaded into Torque --- one simple and one complex. They are placed in the mission such that they are physically intersecting. I've noticed that when the simple DIF comes slightly into view (with complex DIF not in view), the performance behaves as badly as if the complex DIF were in view. So, I am wondering...how does the engine handle visibility and occlusion when two DIFs are intersecting? Why such a performance drop when I am only viewing the simple DIF?
2) Different scenario --- I have one complex DIF. It appears that even when I have a very simple piece of that DIF in view, the performance is not much better than if I were looking at the most complex part. I assumed the performance would be better, considering the whole point of using BSP models is to improve these sorts of visibility/occlusion issues. Why is the engine rendering things that are not in view? (And I am not just talking slightly out of view...but waaaaay out of view.)
I'm trying to put it all together here...When you are looking at a surface of a DIF, does the engine also render all the geometry that is connected to that surface? Or does it render the entire zone that the surface is a part of? What DOES it do?
Please help!
#11
11/15/2006 (3:31 pm)
Are your dif structures properly portalized? That is what allows the bsp mechanism to cull correctly (for the most part).
#12
11/15/2006 (5:27 pm)
Actually in the scenarios I described there was no portalizing going on. I wasn't aware the bsp mechanism relied that much on portals.
#13
11/15/2006 (7:11 pm)
Scott, what do you mean by "complex" dif? high poly? if so, then that's probably the reason you're getting degraded performance. the more faces a dif has, the more time it takes the code to walk through the object and determine what parts visible and whats culled. the proper way to do it would be to use a dif for the main surface or container, and use a smaller dts object for the high detail part. this way, the check determining whether the high detail part should be rendered is just a cheap "bounding box" check, rather than checking polygon planes individually which is what bsp/dif ultimately is doing. 
#14
To completely seal the DIF, one must place walls (brushes) around every side of the structure; even a miniscule gap will cause a leak and no BSP will be generated. Of course if you completely wall up your structure, then you won't be able to enter/exit (and depending on the game this may be what's intended). This is where portals come into play -- portals allow you to completely seal the structure, while still allowing for entrances/exits/windows/etc.
This leads me to a question I've been meaning to ask for a while... Is there any benefit from placing portals internally within the DIF in addition to the outer portals? Something that would give "hinting" to the BSP generation. (Q3 did something like this with doors where it would not render anything behind a door when it was shut.) Any thoughts on this?
Oh and one more thing: Properly portalized DIFs can be set so that terrain is not rendered within the DIF... Although this works correctly, terrain collisions are still checked! This has the effect that you can still run into the terrain inside the DIF even though its invisible. Bug or "feature"? =)
11/16/2006 (12:10 pm)
Regarding portals and BSPs: For the engine to build the BSP tree, the DIF structure must be completely sealed. Otherwise, the DIF is basically rendered just like any DTS object, i.e. no geometry is culled.To completely seal the DIF, one must place walls (brushes) around every side of the structure; even a miniscule gap will cause a leak and no BSP will be generated. Of course if you completely wall up your structure, then you won't be able to enter/exit (and depending on the game this may be what's intended). This is where portals come into play -- portals allow you to completely seal the structure, while still allowing for entrances/exits/windows/etc.
This leads me to a question I've been meaning to ask for a while... Is there any benefit from placing portals internally within the DIF in addition to the outer portals? Something that would give "hinting" to the BSP generation. (Q3 did something like this with doors where it would not render anything behind a door when it was shut.) Any thoughts on this?
Oh and one more thing: Properly portalized DIFs can be set so that terrain is not rendered within the DIF... Although this works correctly, terrain collisions are still checked! This has the effect that you can still run into the terrain inside the DIF even though its invisible. Bug or "feature"? =)
#15
I know difs are better at collision processing. but is there a difference between dts and dif when it comes to non-portalized rendering? is a dif building rendered exactly the same way as a dts building?
11/16/2006 (12:44 pm)
Kevin, what you're saying sounds like the difference between interior hallways, and exterior/outside walls. if I'm understanding you and Stephen correctly, it sounds like dif exteriors(outside faces) don't go through any kind of special culling at all and are essentially rendered just like dts objects. I was under the impression outside faces were frustum culled.I know difs are better at collision processing. but is there a difference between dts and dif when it comes to non-portalized rendering? is a dif building rendered exactly the same way as a dts building?
#16
true, but the TGE engine can be modded to support transparent DIFs. i believe there is at least one resource which does this. i dunno if he used that resource or rolled his own, but Clint added transparent DIFs to the music lounge some time ago.
11/16/2006 (1:42 pm)
Just a quick note -Quote:Transparency is supported on .dif's only in TSE. If you need transparency, you will need to use a .dts.
true, but the TGE engine can be modded to support transparent DIFs. i believe there is at least one resource which does this. i dunno if he used that resource or rolled his own, but Clint added transparent DIFs to the music lounge some time ago.
#17
They are not, Sean. Lighting and culling are very different between the two.
Yuppers, works in the same way as the outer portals. You can seal of a particular room, for example.
11/16/2006 (2:07 pm)
Quote:
I know difs are better at collision processing. but is there a difference between dts and dif when it comes to non-portalized rendering? is a dif building rendered exactly the same way as a dts building?
They are not, Sean. Lighting and culling are very different between the two.
Quote:
This leads me to a question I've been meaning to ask for a while... Is there any benefit from placing portals internally within the DIF in addition to the outer portals? Something that would give "hinting" to the BSP generation. (Q3 did something like this with doors where it would not render anything behind a door when it was shut.) Any thoughts on this?
Yuppers, works in the same way as the outer portals. You can seal of a particular room, for example.
#18
Edit: clarification
11/16/2006 (3:00 pm)
Quote:...it sounds like dif exteriors(outside faces) don't go through any kind of special culling at all and are essentially rendered just like dts objects. I was under the impression outside faces were frustum culled.This is essentially correct -- only the interior spaces are culled via the BSP tree.
Quote:I know difs are better at collision processing. but is there a difference between dts and dif when it comes to non-portalized rendering? is a dif building rendered exactly the same way as a dts building?One could say: Mostly. Like Stefan indicated, lighting is quite different -- DIFs do not support dynamic self/terrain shadowing; DTS models do. DIFs (and surrounding terrain) only receive baked light/shadow maps from the current sunlight settings as well as any light entities placed within the interior structure.
Edit: clarification
#19
@Scott
I stand corrected. this would be the reason your experiencing bad performance with your dif. theres no view frustum culling for outside faces, only zoned interior faces.
according to this thread:www.garagegames.com/blogs/22864/10978 theres already a per-poly frustum culling mechanism available in torque. It probably wouldnt be too hard to use it for your purposes.
11/16/2006 (3:20 pm)
Thanks for the clarification Kevin. @Scott
I stand corrected. this would be the reason your experiencing bad performance with your dif. theres no view frustum culling for outside faces, only zoned interior faces.
according to this thread:www.garagegames.com/blogs/22864/10978 theres already a per-poly frustum culling mechanism available in torque. It probably wouldnt be too hard to use it for your purposes.
#20
11/17/2006 (4:35 am)
Exterior faces are culled when inside a portal, though.
Torque Owner Sven "RaCooN" Knie
two things I can answer.
First for the none seen faces you can use a "null" Texture. This is a texture for the engine to know that the faces don't have to be render. Just make a simple texture (most people make a pink or other bright color) and name it null.jpg or null.png
second smaller textures aren't usefull. The engine creates for it own mipmaps from the textures and if the dif is showing smaller it change to the smaller textures.