Advice- Dynamically merge several DTSs into a single ShapeBase
by Orion Elenzil · in Torque Game Engine · 05/11/2007 (6:17 pm) · 13 replies
Hey All -
i've got a rather daunting task ahead of me and thought i'd post about it here with the hopes that someone may have some advice.
currently our app allows players to dress up their avatars with different clothes. a rather wide variety of clothes. we currently have over 1,200 different clothing items. - many of these items are simply a different texture on the same underlying mesh, but there's also quite a large number of different meshes. i haven't looked at a while, but i'm sure it's at least 300. we use a combo of Mesh-Hiding and Texture Replacement to get all 1,200. ie, all 300 meshes are present in the DTS file for the player, but we only show the five or six which are currently in use. this has worked much, much better than i originally expected, but it's time to get fancier.
for various reasons, primarily the art pipeline but also asset management & distribution,
i need to set it up so that during runtime i can compose a single player ShapeBase object from several different DTS files.
eg
heavyBoots.dts +
razorWireCargoPants.dts +
foofyTutu.dts +
fishnetTankTop.dts +
stockArms.dts +
stockHead.dts +
mohawkHair.dts.
frankly, i'm intimidated.
about a year and a half ago i broached the topic w/ Stephen Zepp and he told me it's a big complex job, and i'm inclined to agree.
i'll jot a few notes.
* a shapebase is composed of several "meshes". i don't need to merge two meshes into one, i just need to dynamically load the meshes it's composed of.
* each mesh has its own set of LODs, so that's nice.
* where will i bury the bones ?
- it'd be nice if each component DTS didn't have a copy of the exact same skeleton nodes in it.
.. that's about as far as i've gotten.
which ain't very far.
anyone know stuff i should be worrying about at night ?
any warnings, detailed advice, vague advice, thumps on the back, etc, much appreciated.
ooo
i've got a rather daunting task ahead of me and thought i'd post about it here with the hopes that someone may have some advice.
currently our app allows players to dress up their avatars with different clothes. a rather wide variety of clothes. we currently have over 1,200 different clothing items. - many of these items are simply a different texture on the same underlying mesh, but there's also quite a large number of different meshes. i haven't looked at a while, but i'm sure it's at least 300. we use a combo of Mesh-Hiding and Texture Replacement to get all 1,200. ie, all 300 meshes are present in the DTS file for the player, but we only show the five or six which are currently in use. this has worked much, much better than i originally expected, but it's time to get fancier.
for various reasons, primarily the art pipeline but also asset management & distribution,
i need to set it up so that during runtime i can compose a single player ShapeBase object from several different DTS files.
eg
heavyBoots.dts +
razorWireCargoPants.dts +
foofyTutu.dts +
fishnetTankTop.dts +
stockArms.dts +
stockHead.dts +
mohawkHair.dts.
frankly, i'm intimidated.
about a year and a half ago i broached the topic w/ Stephen Zepp and he told me it's a big complex job, and i'm inclined to agree.
i'll jot a few notes.
* a shapebase is composed of several "meshes". i don't need to merge two meshes into one, i just need to dynamically load the meshes it's composed of.
* each mesh has its own set of LODs, so that's nice.
* where will i bury the bones ?
- it'd be nice if each component DTS didn't have a copy of the exact same skeleton nodes in it.
.. that's about as far as i've gotten.
which ain't very far.
anyone know stuff i should be worrying about at night ?
any warnings, detailed advice, vague advice, thumps on the back, etc, much appreciated.
ooo
About the author
#2
You'd have to cobble together how the submeshes get animated (by tracking the skeleton nodes or something similiar). It depends on your needs there.
Alternatively, you might be able to hack the DTS loading code to allow you to load submeshes out of the file and replace it in a TSShape. (Sounds like more what you wanted to do)
Hope that helps a little.
05/16/2007 (9:55 am)
If you just need a starting point, look at the wheeledvehicle object. The wheels are not part of the base shape and are rendered custom in the wheeledvehicle class. If I had to do this, that's at least where I'd start.You'd have to cobble together how the submeshes get animated (by tracking the skeleton nodes or something similiar). It depends on your needs there.
Alternatively, you might be able to hack the DTS loading code to allow you to load submeshes out of the file and replace it in a TSShape. (Sounds like more what you wanted to do)
Hope that helps a little.
#3
05/16/2007 (9:57 am)
I wish you luck, and then for a resource when it's done! ;)
#4
Yeah, i'm hoping to keep things tidy so it can be a resource.
it's tricky sometimes because the codebase i'm using has drifted fairly far away from stock,
but this seems pretty generic so hopefully.
Brian -
i'll have a look at wheeled vehicle. i didn't know the wheels were seperate objects.
however i'm definitely looking for something which only touches the object-creation process
versus digging into rendering etc. ie, once the ShapeBase is constructed, all the regular skeleton and mesh-manipulation and texture replacement stuff etc etc works on it unchanged.
05/16/2007 (10:22 am)
Hey Guys - thanks for the words.Yeah, i'm hoping to keep things tidy so it can be a resource.
it's tricky sometimes because the codebase i'm using has drifted fairly far away from stock,
but this seems pretty generic so hopefully.
Brian -
i'll have a look at wheeled vehicle. i didn't know the wheels were seperate objects.
however i'm definitely looking for something which only touches the object-creation process
versus digging into rendering etc. ie, once the ShapeBase is constructed, all the regular skeleton and mesh-manipulation and texture replacement stuff etc etc works on it unchanged.
#5
If you go with modifiying the rendering process, you'll be able to share that data easier. I'd have a base skeleton object that animated and basically glue the subobjects onto the skeleton nodes. Again, the wheeled vehicle is a good starting point (as is the mounted image code).
In both cases, I doubt the texture replace stuff will work unchanged. It's based off of the parent shape name and would require that your texture coordianates are set up "just right". Maybe your art is already there and this is a non-issue.
Anyways, just some random things to think about.
05/16/2007 (11:42 am)
Yeah, I'd see how modifying the loading stuff is attractive. But you'll have to worry about things like not being able to reuse that TSShape among instances of characters. So if you have 40 characters in a room, all with mohawk hair, you'll have 40 instances of the vertices, texture coordinates, and other data that is normally shared through the TSShape. If you go with modifiying the rendering process, you'll be able to share that data easier. I'd have a base skeleton object that animated and basically glue the subobjects onto the skeleton nodes. Again, the wheeled vehicle is a good starting point (as is the mounted image code).
In both cases, I doubt the texture replace stuff will work unchanged. It's based off of the parent shape name and would require that your texture coordianates are set up "just right". Maybe your art is already there and this is a non-issue.
Anyways, just some random things to think about.
#6
yeah, that's pretty much necessary since it needs to handle 50 - 200 players on screen at once.
hm actually i wonder if it is necessary.
sharing the resources would save memory but not CPU, afaik.
and the bottleneck is mesh deformation for skin.
with as many clothing variations as we have,
the amount of meshes shared even by say 200 players might only be twenty or so.
hm.
this is exactly what i was hoping for, discussion-wise, thanks!
05/16/2007 (11:55 am)
Ooh, excellent point about shared resources.yeah, that's pretty much necessary since it needs to handle 50 - 200 players on screen at once.
hm actually i wonder if it is necessary.
sharing the resources would save memory but not CPU, afaik.
and the bottleneck is mesh deformation for skin.
with as many clothing variations as we have,
the amount of meshes shared even by say 200 players might only be twenty or so.
hm.
this is exactly what i was hoping for, discussion-wise, thanks!
#7
look at TSShapeConstructor which is used to combine DTS and DSQ files into one object in Torque.
Together with WheeledVehicle it can (hopefully) lead you to right direction.
05/16/2007 (1:05 pm)
Orion, just a thought:look at TSShapeConstructor which is used to combine DTS and DSQ files into one object in Torque.
Together with WheeledVehicle it can (hopefully) lead you to right direction.
#8
about your
So, let's imagine we have DTS:
1. body.dts - with all nodes of the "torso" and "neck" node in it.
2. head.dts - with all "head" nodes in it, and "neck" node too. Also includes hatMountPoint node.
3. hat.dts - with only single node hatMountPoint
So, making something like this:
Now, the question - how :) good luck and looking forward for progress on it.
edit: code formatting
05/16/2007 (1:14 pm)
And, another "note":about your
Quote:* where will i bury the bones ?I think that using something like a "mount" (wheeledvehicle) but in a TSShapeConstructor-way can be useful.
- it'd be nice if each component DTS didn't have a copy of the exact same skeleton nodes in it.
So, let's imagine we have DTS:
1. body.dts - with all nodes of the "torso" and "neck" node in it.
2. head.dts - with all "head" nodes in it, and "neck" node too. Also includes hatMountPoint node.
3. hat.dts - with only single node hatMountPoint
So, making something like this:
datablock TSShapeConstructorPlus(myNewDTS)
{
baseShape = "./torso.dts torso"; // dts file and "internal name" of it
addShape0 = "./head.dts head torso neck"; // dts file, internal name, internal name to mount to, node to use
addShape1 = "./hat.dts hat head hatMountPoint"; // dts file, internal name, internal name to mount to, node to use
/// etc, etc, etc
sequence0 = "./anim_root.dsq root";
sequence1 = "./anim_forward.dsq run";
/// etc, etc, etc
};can theoretically give possibility to combine multiple meshes into one single "in-torque" object.Now, the question - how :) good luck and looking forward for progress on it.
edit: code formatting
#9
we've decided to try an approach where we break the current requirement that each TSShapeInstance correspond to only one TSShape. ie, a single TSShapeInstance will now reference multiple TSShapes.
we'll still have hundreds of meshes in the player object, all hidden except for a handful, but we'll also have hundreds of DTS files instead of just one. which strange as it may sound is what we want.
07/05/2007 (11:47 am)
Fwiw -we've decided to try an approach where we break the current requirement that each TSShapeInstance correspond to only one TSShape. ie, a single TSShapeInstance will now reference multiple TSShapes.
we'll still have hundreds of meshes in the player object, all hidden except for a handful, but we'll also have hundreds of DTS files instead of just one. which strange as it may sound is what we want.
#10
I haven't looked at it at all myself, but breaking the assumption implies to me a quick balooning of your total memory footprint, since the TSShapeInstances cannot necessarily be treated in a "reference count" manner within the resource manager.
Of course forewarned (or for-known if you guys caught that!), you could probably re-work the resource manager instancing to meet your footprint needs ;)
07/05/2007 (4:09 pm)
Not suggesting that your direction is a bad one, but be careful of breaking that assumption--or at least follow through the resource manager and what it does.I haven't looked at it at all myself, but breaking the assumption implies to me a quick balooning of your total memory footprint, since the TSShapeInstances cannot necessarily be treated in a "reference count" manner within the resource manager.
Of course forewarned (or for-known if you guys caught that!), you could probably re-work the resource manager instancing to meet your footprint needs ;)
#11
as always thanks for your comments;
i'll pass them on to the fellow who's actually implementing this.
do you mean we'll hit trouble w/ the ref counts on the TSShapeInstances or the TSShapes ?
to repeat (partially for my own clarification),
the stock TGE relationship is Many TSShapeInstances to One TSShape,
and we're changing it to Many to Many.
07/05/2007 (4:22 pm)
Hey Stephen -as always thanks for your comments;
i'll pass them on to the fellow who's actually implementing this.
do you mean we'll hit trouble w/ the ref counts on the TSShapeInstances or the TSShapes ?
to repeat (partially for my own clarification),
the stock TGE relationship is Many TSShapeInstances to One TSShape,
and we're changing it to Many to Many.
#12
When I read your summary, the first thought that came to my head was "hmm...Torque loads one copy of a dts normally and then uses the normal resource manager to allow references. I wonder if breaking that assumption also breaks the way that works?"
My suggestion would be to prototype implement, watch your memory footprint, and if it inflated, check the ref count stuff regarding loading dts shapes and the resource manager.
07/05/2007 (4:30 pm)
I don't know for certain if it will be an issue at all to be honest--just something to watch for.When I read your summary, the first thought that came to my head was "hmm...Torque loads one copy of a dts normally and then uses the normal resource manager to allow references. I wonder if breaking that assumption also breaks the way that works?"
My suggestion would be to prototype implement, watch your memory footprint, and if it inflated, check the ref count stuff regarding loading dts shapes and the resource manager.
#13
I'm keeping a close eye on the Resource Manager, so far not too much trouble. I've derived a class from TSShapeInstance that keeps track of multiple TSShapes. The ShapeBase class (and related classes) now have a member of the new class type, and a couple of optional fields go into the datablock; if multiple files are provided (example: player), it is handled in the new class implementation, and if it is a 'normal' ShapeBase-derived class the new class simply passes calls directly through to TSShapeInstance. So far so good!
Thanx...
Sean
07/05/2007 (5:21 pm)
Stephen,I'm keeping a close eye on the Resource Manager, so far not too much trouble. I've derived a class from TSShapeInstance that keeps track of multiple TSShapes. The ShapeBase class (and related classes) now have a member of the new class type, and a couple of optional fields go into the datablock; if multiple files are provided (example: player), it is handled in the new class implementation, and if it is a 'normal' ShapeBase-derived class the new class simply passes calls directly through to TSShapeInstance. So far so good!
Thanx...
Sean
Associate Orion Elenzil
Real Life Plus