Sharing Animations with Multiple Models
by Cameron Tofer · in Torque Game Engine · 06/15/2005 (9:27 pm) · 6 replies
Hi there, we have several different human models that all use the same skeleton. We have one set of animations that all the characters use. Using the tsShapeContructor we can link all the animations to each model. Everything so far works. great.
Now for fun: While trying to speed up our load times and decrease our memory requirements i've noticed that the tsShapeContructor streams the animation from disk and copies it into the TSShape, regardless if that animation has been loaded or not. so here's what i have:
10-15 different models sharing roughly 70 or so animations. so each model has a tsShapeConstructor and its loading and copying each animation. each animation is loaded 10 times. and i guess its twice again if you're single player? ( loads for each client and server, i might be wrong here just guessing ).
anyhow i would like to make the dsq file a resource and load all the animations at the start in one go and then have all the characters simply point to the data. I haven't done too much searching yet on how much of a pain this would be. thoughts?
Now for fun: While trying to speed up our load times and decrease our memory requirements i've noticed that the tsShapeContructor streams the animation from disk and copies it into the TSShape, regardless if that animation has been loaded or not. so here's what i have:
10-15 different models sharing roughly 70 or so animations. so each model has a tsShapeConstructor and its loading and copying each animation. each animation is loaded 10 times. and i guess its twice again if you're single player? ( loads for each client and server, i might be wrong here just guessing ).
anyhow i would like to make the dsq file a resource and load all the animations at the start in one go and then have all the characters simply point to the data. I haven't done too much searching yet on how much of a pain this would be. thoughts?
#2
it calls:
f = ResourceManager->openStream(fileBuf);
...
if (!shape->importSequences(f) || f->getStatus()!=Stream::Ok)
...
ResourceManager->closeStream(f);
openStream goes straight to the disk everytime. I know toque has a ResourceManager and I would like to DSQ's and other file types to use it. but there's going to be some work fixing all this stuff. My real concern is how much work TSShape is going to be to modifiy so it can point to a series of DSQ file resources.
p.s. all the animation loading happens before the loading bar. i was wondering why the harddrive was going crazy before the loading bar would pop up!
06/16/2005 (8:54 am)
Yes, i'm 100% sure. try looking at TSShapeConstructor::preloadit calls:
f = ResourceManager->openStream(fileBuf);
...
if (!shape->importSequences(f) || f->getStatus()!=Stream::Ok)
...
ResourceManager->closeStream(f);
openStream goes straight to the disk everytime. I know toque has a ResourceManager and I would like to DSQ's and other file types to use it. but there's going to be some work fixing all this stuff. My real concern is how much work TSShape is going to be to modifiy so it can point to a series of DSQ file resources.
p.s. all the animation loading happens before the loading bar. i was wondering why the harddrive was going crazy before the loading bar would pop up!
#3
This sounds like a worthwhile project, so I'd be happy to give advice to help you get going on it.
06/16/2005 (10:32 am)
That is how the TSShapeConstructor works. It's mostly a convenience to allow sharing of animation data, not a memory optimization. Improving it to be such would be pretty cool, though. Maybe adding a TSAnimationSet resource type, and then storing a vector of Resource<> handles in the TSShapeInstance or TSShape, overlaying the animations by checking the vector?This sounds like a worthwhile project, so I'd be happy to give advice to help you get going on it.
#4
1) Streams need to be identifiable by source. "This stream reffers to that file" kind of thing. Could be a string (file path), or a number, but should be in the base stream class (to return some BAD_ID), and then fixed wherever is relevant (FileStream) to return meaningful info.
2) importSequences() could then check a static (or at least per-mission) cache before loading the animation data.
You could then move the animation info out of TSShape ("Vector sequences;" in TSShape.h according to dOxygen) and into a StreamID->Vector map of some sort. Turn "sequences" a pointer in TSShape, adjust the various 'sequence' uses to treat it as a pointer, and be good to go.
Unless there's more to animation data than just those Sequences. But given my Highly Limited Knowledge, it's possible that I'm missing a thing or two hundred.
---
EDIT:
For example, I might have missed that you've got different animations in different files. This proposal would only cover a case where all the animations were shared, and all were from the same source.
To get that to work, you'd have to handle multiple Vector's in the same shape.
OTOH, a Sequence could be a pile of vertex info specific to the model in question, rather than the bones, in which case all bets are off.
More digging revealed that all the data in TSShape (and its structs) are just indexes to Actual Data stored elsewhere... so having big piles of Sequences isn't anywhere near as bad as one might think at first.
---
EDIT 2: Okay, it's official. I don't have a clue how TGE's animation system stores its data, despite having been through the dOxygen files for the last 30 miniutes.
06/16/2005 (1:22 pm)
I would think you could add some special sauce to Stream and importSequences:1) Streams need to be identifiable by source. "This stream reffers to that file" kind of thing. Could be a string (file path), or a number, but should be in the base stream class (to return some BAD_ID), and then fixed wherever is relevant (FileStream) to return meaningful info.
2) importSequences() could then check a static (or at least per-mission) cache before loading the animation data.
You could then move the animation info out of TSShape ("Vector
Unless there's more to animation data than just those Sequences. But given my Highly Limited Knowledge, it's possible that I'm missing a thing or two hundred.
---
EDIT:
For example, I might have missed that you've got different animations in different files. This proposal would only cover a case where all the animations were shared, and all were from the same source.
To get that to work, you'd have to handle multiple Vector
OTOH, a Sequence could be a pile of vertex info specific to the model in question, rather than the bones, in which case all bets are off.
More digging revealed that all the data in TSShape (and its structs) are just indexes to Actual Data stored elsewhere... so having big piles of Sequences isn't anywhere near as bad as one might think at first.
---
EDIT 2: Okay, it's official. I don't have a clue how TGE's animation system stores its data, despite having been through the dOxygen files for the last 30 miniutes.
#5
2) Gotta have a stack of animation sets, which you can then search through. Getting that to happen will be a trick.
06/16/2005 (2:43 pm)
1) I have no idea how that would help. For the cases where it matters that info can be had...2) Gotta have a stack of animation sets, which you can then search through. Getting that to happen will be a trick.
#6
06/16/2005 (10:29 pm)
Ok, it looks like a bunch of vectors in TSShape need to be moved to a new animation object. but it also seems logical to not store any of this stuff in the .dts file. so for .dts files that have sequences it could be a .dts with .dsq streamed after it. 
Torque 3D Owner Stephen Zepp
I'm betting that it's not actually "loading" them for each and every model, but simply interacting with the resource manager to make sure they are linked to each of the models appropriately.