Torque 3D Development - Chris Robertson and Collada
by Brett Seyler · 02/17/2009 (10:19 am) · 84 comments
Last week we introduced the first look at at new tools for Torque 3D with the Torque Launcher. Today, we've got a very big announcement to follow that up. Though we've already talked about this some publicly in IRC and the forums, it's going to radically change Torque's content pipeline.
Chris' Bio
"I'm a software developer from New Zealand. Been using Torque in my spare time for about 5 years. Best known for developing the Milkshape and Houdini DTS exporters.""I'm working on adding Collada support to Torque, so the engine can now load a Collada model in the same way it loads a DTS (ie. as part of a script object or directly from the World Editor). I've also extended TSShapeConstructor to allow nodes, sequences and even meshes to be added, removed and renamed when a DTS or Collada shape is loaded. This is a great complement to the Collada loader, since you can take just about any model at all (such as from Google's 3D Warehouse) and "Torquify" it without having to muck around with importing and re-exporting."
Apparently New Zealand is too small for Google to acknowdege, but it's the land of Mordor and Flight of the Concords, so it's already more famous that Australia (though I am a Russell Crowe fan). Kiwis have better accents too.
Now, we've had this working with Torque for some time now, but this is the kind of feature that's going to take many, many use cases before we're satisfied that it's 100%. To that end, we're inviting everyone to test this functionality with any and all content you can find and give us your feedback. Let us know where we still need to work out some kinks, and by the time we release Torque 3D, it will have the smoothest, easiest, most polished art pipeline out there for the widest number of tools.
How can you get started?
We've created a custom build of TGEA 1.8.1 with the Collada loading functionality wrapped in. This is a binary build, but it should provide everything you need to test out the new content pipeline. You'll find thorough documentation in the root install directory, and we're encouraging you to post your feedback in this forum. You can also read more technical details in this forum post. The Torque Collada Test is set to expire in less than a month (March 13th at 5:00 PM PST) so jump in as soon as possible and start testing!
Download Torque Collada Test
I had a quick run at what we could do with this new build, so I'll share it with you all as a quick walk-though.
Collada loading in Torque is a big deal, but it's one of many pieces that will change the way artists work with Torque 3D. There's a lot more to show. Please take some time and take the Torque 3D Collada Test build for a spin. Help us make Torque 3D even better with your feedback!
More development blogs to come. This is post #10.
Torque 3D development blogs:
- Post #1 - Kickoff
- Post #2 - Apparatus and Warrior Camp
- Post #3 - Luma's racing kit
- Post #4 - Josh Engebretson and Web Publishing
- Post #5 - Pricing and Licensing
- Post #6 - Pricing and Licensing CONTINUED
- Post #7 - Wetness & Precipitation
- Post #8 - Screeen Space Ambient Occlusion (SSAO)
- Post #9 - Matt Langley and the Torque Launcher
- Post #10 - Chris Robertson and Collada
- Post #11 - Depth of Field
- Post #12 - Advanced Lighting
- Post #13 - Soft Particles
- Post #14 - World Editor
- Post #15 - Pricing and Licensing ANNOUNCED!
- Post #16 - GDC Live Edition
- Post #17 - River & Road Editors
- Post #18 - Beta is UP!
- Post #19 - Light Rays, Undercity, Material Editor
- Post #20 - Mass Market Hardware
- Post #21 - Beta: Part Deux
- Post #22 - Marching Towards Beta 3
- Post #23 - pureLIGHT
- Post #24 - Lighting, Terrain, and Cloth
- Post #25 - Beta 3! [li]Post #26 - Coming Soon!
About the author
Since 2007, I've done my best to steer Torque's development and brand toward the best opportunities in games middleware.
#22
I am referring to ShowToolPro and not only to ShowToolPro.
I don't unterstand how i can adjust things like names of animations, LODs, Portals, Vehicle Setups, Billboards, Collision Meshes and so on.
( David Wyand's DTS-Exporter for Lightwave is the best exporter i have ever seen )
02/17/2009 (7:05 pm)
@Michael Perry: Quote:
@Thomas - We contracted Chris to work on the Collada loader and TSShapeConstructor, not improve STP. We are showing off the fruits of his labor and I think it is pretty damn cool.
STP was never meant to be a refinement tool. ShowToolPro. There are some other features going into Torque 3D that will cover your concerns. Try not to jump to conclusions until you read the rest of Brett's blogs.
I am referring to ShowToolPro and not only to ShowToolPro.
I don't unterstand how i can adjust things like names of animations, LODs, Portals, Vehicle Setups, Billboards, Collision Meshes and so on.
( David Wyand's DTS-Exporter for Lightwave is the best exporter i have ever seen )
#23
The docs that come with the Collada Test cover all of this.
02/17/2009 (7:10 pm)
Quote:
I don't unterstand how i can adjust things like names of animations, LODs, Portals, Vehicle Setups, Billboards, Collision Meshes and so on.
The docs that come with the Collada Test cover all of this.
#24
@Names of animation:
Go read the docs.
02/17/2009 (7:11 pm)
Going off Matt's Post:@Names of animation:
renameSequence(string, string)
This command is used to rename an animation sequence.
Syntax
renameSequence(string old_name, string new_name)
* old_name: old sequence name
* new_name: new name of the sequence
Returns
No return value.
Examples
%this.renameSequence("walking", "walk");Go read the docs.
#25
For example: you can script together a 'Player' type avatar that will load the SDK DSQ library, from 'nothing' but script; how cool is that?
Here is an example from the .pdf of the new Constructor Class and functions:
Hopefully this parsed well enough to make the point that with this new 'Collada' Read[I think 'overSimpliying things??, there is much more going on now behind the 'scene']...comes some very useful tools to manipulate the shapes working inside the engine. It's only now, that the Editors simply read DAE format, another useful tidbit as you protoType away with whatever asset you can get your hands on!
Cheers!
I think there may have been some misconceptions of this new 'read' of Collada; as there is also the new TSShapeConstructor going on as well. That is the real story here....you can construct shapes for the engine from more than DTS/DSQ formats for geometry and animation! Collada read is a nice byproduct, lol?!? Think of it that way to make everyone more comfy.:)
I was there in the very-very beginnings, and suggested some expansion of the Class as I was getting some widely varying 'flavours' of DAE from several DCC's...not to steal any 'thunder' from the real 'behind the Scene' guy....take a grand bow, Chris, you deserve it!!
02/17/2009 (7:49 pm)
A full reading of the 'new' TSShapeConstructor class should be done. You can do a vast number of 'shape-related' functions, as there are new 'functions' that deal with some very cool shape manipulation.For example: you can script together a 'Player' type avatar that will load the SDK DSQ library, from 'nothing' but script; how cool is that?
Here is an example from the .pdf of the new Constructor Class and functions:
// Create a simple (rigid) player model using boxes for body parts
function CreateRigidPlayer(%shape)
{
CreateCubeObject(%shape, "head", "Eye", "0.2 0.2 0.3");
CreateCubeObject(%shape, "upper_torso", "Bip01 Spine2", "0.3 0.2 0.4");
CreateCubeObject(%shape, "lower_torso", "Bip01 Spine", "0.3 0.2 0.35");
CreateCubeObject(%shape, "left_thigh", "Bip01 L Thigh", "0.5 0.2 0.2 0.25 0 0");
CreateCubeObject(%shape, "left_leg", "Bip01 L Calf", "0.4 0.2 0.2 0.25 0 0");
CreateCubeObject(%shape, "left_foot", "Bip01 L Foot", "0.2 0.2 0.2");
CreateCubeObject(%shape, "right_thigh", "Bip01 R Thigh", "0.5 0.2 0.2 0.25 0 0");
CreateCubeObject(%shape, "right_leg", "Bip01 R Calf", "0.4 0.2 0.2 0.25 0 0");
CreateCubeObject(%shape, "right_foot", "Bip01 R Foot", "0.2 0.2 0.2");
CreateCubeObject(%shape, "left_upper_arm", "Bip01 L UpperArm", "0.3 0.15 0.15 0.15 0 0");
CreateCubeObject(%shape, "left_forearm", "Bip01 L Forearm", "0.2 0.15 0.15 0.1 0 0");
CreateCubeObject(%shape, "left_hand", "Bip01 L Hand", "0.15 0.15 0.15");
CreateCubeObject(%shape, "right_upper_arm", "Bip01 R UpperArm", "0.3 0.15 0.15 0.15 0 0");
CreateCubeObject(%shape, "right_forearm", "Bip01 R Forearm", "0.2 0.15 0.15 0.1 0 0");
CreateCubeObject(%shape, "right_hand", "Bip01 R Hand", "0.15 0.15 0.15");
// create collision mesh
%shape.addMesh("cube", "bounds", "Col-1");
}Hopefully this parsed well enough to make the point that with this new 'Collada' Read[I think 'overSimpliying things??, there is much more going on now behind the 'scene']...comes some very useful tools to manipulate the shapes working inside the engine. It's only now, that the Editors simply read DAE format, another useful tidbit as you protoType away with whatever asset you can get your hands on!
Cheers!
I think there may have been some misconceptions of this new 'read' of Collada; as there is also the new TSShapeConstructor going on as well. That is the real story here....you can construct shapes for the engine from more than DTS/DSQ formats for geometry and animation! Collada read is a nice byproduct, lol?!? Think of it that way to make everyone more comfy.:)
I was there in the very-very beginnings, and suggested some expansion of the Class as I was getting some widely varying 'flavours' of DAE from several DCC's...not to steal any 'thunder' from the real 'behind the Scene' guy....take a grand bow, Chris, you deserve it!!
#26
I guess you dont want to understand me. I give up. Do what you want.
02/17/2009 (8:05 pm)
@Matt and Michael: I guess you dont want to understand me. I give up. Do what you want.
#27
For years, the community as a whole has been asking for a better and easier to use art pipeline. We are finally offering that.
We are doing what we want, and part of what we want is to address the major issues expressed by the community for years.
We aren't ignoring you, but if you can't handle our suggestions, advice, and our logic...then fine. But you should at least try the system out before passing judgement.
02/17/2009 (8:41 pm)
We are hearing you, and we are trying to explain how the loader, TSShapeConstructor, and some of the other features address your concerns. Have you tried the test and read the docs? We want your feedback, but we need feedback based on the testing and not speculation.For years, the community as a whole has been asking for a better and easier to use art pipeline. We are finally offering that.
We are doing what we want, and part of what we want is to address the major issues expressed by the community for years.
We aren't ignoring you, but if you can't handle our suggestions, advice, and our logic...then fine. But you should at least try the system out before passing judgement.
#28
I made the call to not handle our Collada importing process through an external application. I did so for a variety of reasons:
1) Updating ShowTool Pro to do more than display a model and its information is actually a pretty hefty undertaking. I know this from direct experience with working on Constructor with Dave Wyand (who did ShowTool Pro and brought over code from it). I would estimate that it could take as long as 6-9 months to build a proper "conditioner" and that isn't even touching on displaying and editing shaders. This is an investment of resources that I do not think is worth it in the end.
2) I really, really want Torque to have an art pipeline that "just works" out of the box for basic stuff. You shouldn't have to read 6 pages of documents to get a simple box to show up on the screen (bare minimum you need 3 nodes and a bounds with the DTS exporters). Likewise, I think it is silly to force the user to use a separate, stand-alone application to get that box into the engine. Instead, I would rather rely on industry standard exporters that ship with nearly every art tool out there (no more waiting on someone to create a DTS exporter) which can talk directly and easily to the engine and getting your "box" into the game is done with no extra complexities.
3) History and experience has shown that keeping an external tool up-to-date with the engine is nearly impossible (ShowTool, Constructor, etc). I would much rather spend our precious resources on improving the in-game tools and engine rather than on maintenance especially when those improvements could cover the same functionality that the external tool does.
Currently the Collada pipeline supports pretty much all of the conditioning that you could want through naming conventions and the newly exposed script functions. Yes, we are aware that it is a little clunky to deal with these purely from script and we are actively looking at ways to make that easier.
02/17/2009 (9:35 pm)
Thomas,I made the call to not handle our Collada importing process through an external application. I did so for a variety of reasons:
1) Updating ShowTool Pro to do more than display a model and its information is actually a pretty hefty undertaking. I know this from direct experience with working on Constructor with Dave Wyand (who did ShowTool Pro and brought over code from it). I would estimate that it could take as long as 6-9 months to build a proper "conditioner" and that isn't even touching on displaying and editing shaders. This is an investment of resources that I do not think is worth it in the end.
2) I really, really want Torque to have an art pipeline that "just works" out of the box for basic stuff. You shouldn't have to read 6 pages of documents to get a simple box to show up on the screen (bare minimum you need 3 nodes and a bounds with the DTS exporters). Likewise, I think it is silly to force the user to use a separate, stand-alone application to get that box into the engine. Instead, I would rather rely on industry standard exporters that ship with nearly every art tool out there (no more waiting on someone to create a DTS exporter) which can talk directly and easily to the engine and getting your "box" into the game is done with no extra complexities.
3) History and experience has shown that keeping an external tool up-to-date with the engine is nearly impossible (ShowTool, Constructor, etc). I would much rather spend our precious resources on improving the in-game tools and engine rather than on maintenance especially when those improvements could cover the same functionality that the external tool does.
Currently the Collada pipeline supports pretty much all of the conditioning that you could want through naming conventions and the newly exposed script functions. Yes, we are aware that it is a little clunky to deal with these purely from script and we are actively looking at ways to make that easier.
#29
You can already add, remove, and rename nodes in your modeling application and just save it back out. In fact, you could even open the Collada file up in a text editor and make those changes (easier to do it in you modeling application but it is an option).
From my perspective really the only difficult bit is defining the Torque "sequence" information (start frame, end frame, cyclic, blend, etc) since, from what I've heard, most of the Collada exporters out there aren't actively supporting the "animation clips" part of the spec. This is the area that we are looking at most closely.
Is there something else that has you concerned?
02/17/2009 (9:48 pm)
As a follow up...I am curious exactly which bits of functionality you think would have to be covered by an external Collada conditioner that isn't covered already by your modeling application?You can already add, remove, and rename nodes in your modeling application and just save it back out. In fact, you could even open the Collada file up in a text editor and make those changes (easier to do it in you modeling application but it is an option).
From my perspective really the only difficult bit is defining the Torque "sequence" information (start frame, end frame, cyclic, blend, etc) since, from what I've heard, most of the Collada exporters out there aren't actively supporting the "animation clips" part of the spec. This is the area that we are looking at most closely.
Is there something else that has you concerned?
#30
And Michael, Matt & Brett, a bunch of ye folks are actually doing a great job keeping things together, strategies transparent and engines alive.
Thanks & keep it up :)
02/17/2009 (10:28 pm)
Quote:For years, the community as a whole has been asking...
, and part of what we want is to address the major issues expressed by the community for years.
And Michael, Matt & Brett, a bunch of ye folks are actually doing a great job keeping things together, strategies transparent and engines alive.
Thanks & keep it up :)
#31
I know what the problem is. It is David Wyand's DTS Exporter for Lightwave.
When i use this nice tool i dont have to care about nodes and the naming conventions. I just do it with my mouse.
This geometry is a visible mesh at LOD0. And this geometry is a Collision Mesh at LOD0, changing LOD0 to LOD1 at pixel size XY, this is a billboard, there is animation "walk" from frame X to frame Y, there is animation "jump" from frame X to frame Y and so on.
I just do that with a few clicks.
But now Collada. I dont know how to set up my Scene (Hierarchy) with Lightwave's Scene Editor. I dont know exactly the name of each special node. That's the problem.
I found this document but the information goes not deep enough.
( http://fosters.realmwarsgame.com/tools/DTS_Nodes.pdf )
So i thought it is a good idea to improve ShowToolPro ( assign special nodes with a few clicks ).
So my suggestion is to explain all the nodes we, the user, need to describe a model.
But there are three more problems.
Problem 2: Interiors and Portals
I dont know something about the future of Interiors (dif). Will T3D support dif's or do i have to create Interiors out of models? If so, how do i have to create portals? Are there special nodes?
Problem 3: Animated Collision Meshes
I know it from TGE. Animated Collision Meshes are showed by ShowToolPro but they are not calculated (checked) by TGE. For Example... open/close doors are a common problem.
Problem 4: Self-Shadowing of Models
I tried the "Torque Collada Test" and noticed the Shadows from these Collada's on the Terrain. But these Shadows are not received by other static models.
Dont get me wrong. I like Torque and i would buy T3D (assumed a fair price) but the problems described above are a real nightmare for me.
02/17/2009 (10:43 pm)
@Matt Fairfax:I know what the problem is. It is David Wyand's DTS Exporter for Lightwave.
When i use this nice tool i dont have to care about nodes and the naming conventions. I just do it with my mouse.
This geometry is a visible mesh at LOD0. And this geometry is a Collision Mesh at LOD0, changing LOD0 to LOD1 at pixel size XY, this is a billboard, there is animation "walk" from frame X to frame Y, there is animation "jump" from frame X to frame Y and so on.
I just do that with a few clicks.
But now Collada. I dont know how to set up my Scene (Hierarchy) with Lightwave's Scene Editor. I dont know exactly the name of each special node. That's the problem.
I found this document but the information goes not deep enough.
( http://fosters.realmwarsgame.com/tools/DTS_Nodes.pdf )
So i thought it is a good idea to improve ShowToolPro ( assign special nodes with a few clicks ).
So my suggestion is to explain all the nodes we, the user, need to describe a model.
But there are three more problems.
Problem 2: Interiors and Portals
I dont know something about the future of Interiors (dif). Will T3D support dif's or do i have to create Interiors out of models? If so, how do i have to create portals? Are there special nodes?
Problem 3: Animated Collision Meshes
I know it from TGE. Animated Collision Meshes are showed by ShowToolPro but they are not calculated (checked) by TGE. For Example... open/close doors are a common problem.
Problem 4: Self-Shadowing of Models
I tried the "Torque Collada Test" and noticed the Shadows from these Collada's on the Terrain. But these Shadows are not received by other static models.
Dont get me wrong. I like Torque and i would buy T3D (assumed a fair price) but the problems described above are a real nightmare for me.
#32
This doesn't help as much with the animation sequences but I think I've already covered that in detail.
Honestly though, if you are perfectly happy with your DTS exporter then why do you care one way or another about the Collada loader? This isn't designed to replace DTS files. It is intended to add an additional option to the Torque art pipeline that we think will be very useful to new users who aren't comfortable with DTS exporting.
As far as Problems 2-4...these have nothing to do with the Collada loader. Those are separate feature requests for the engine itself. If you would like, you can document them in the What would you like to see addressed in Torque 3D?.
02/17/2009 (11:10 pm)
I haven't used Lightwave or Dave's exporter but I believe that it actually creates and renames the nodes and meshes for you in the actual scene so that they will export correctly (so if you have a mesh named "foo" and you specify it as detail level 2 it will rename the mesh to "foo2" and add the "detail2" node to the scene). If this is how Dave's exporter works then you should be able to use it with the Collada exporter (just don't save to DTS).This doesn't help as much with the animation sequences but I think I've already covered that in detail.
Honestly though, if you are perfectly happy with your DTS exporter then why do you care one way or another about the Collada loader? This isn't designed to replace DTS files. It is intended to add an additional option to the Torque art pipeline that we think will be very useful to new users who aren't comfortable with DTS exporting.
As far as Problems 2-4...these have nothing to do with the Collada loader. Those are separate feature requests for the engine itself. If you would like, you can document them in the What would you like to see addressed in Torque 3D?.
#33
Problems 2-4: I dont want to write these requests again. You now know about.
If you dont want to say something about... it is ok... i will wait for some new posts from Brett Seyler.
02/17/2009 (11:45 pm)
I did not know that Collada is "only" a option. I thought that Collada is the one and only format in the future. Problems 2-4: I dont want to write these requests again. You now know about.
If you dont want to say something about... it is ok... i will wait for some new posts from Brett Seyler.
#34
This blog isn't what Matt uses to track Torque 3D requests and feedback. The thread he linked to is. Much like my documentation feedback threads, if someone makes a suggestion about docs I ask them to please double post it in those threads.
It's organized, it's convenient, it's subscribed to, and it's a simple request. If your input is lost, then part of that will be because it wasn't logged in an official (and stickied) thread.
02/17/2009 (11:52 pm)
I'm sure someone is going to poke me and tell me to let it go, but oh well...Quote:Problems 2-4: I dont want to write these requests again. You now know about.
This blog isn't what Matt uses to track Torque 3D requests and feedback. The thread he linked to is. Much like my documentation feedback threads, if someone makes a suggestion about docs I ask them to please double post it in those threads.
It's organized, it's convenient, it's subscribed to, and it's a simple request. If your input is lost, then part of that will be because it wasn't logged in an official (and stickied) thread.
#35
02/18/2009 (12:49 am)
@GG guys: This is looking fantastic! There are too many hoops to jump through with the current (DTS) pipeline -- anything that expedites the art pipeline is a welcome addition! =)
#36
02/18/2009 (12:51 am)
Does that mean if I ship a content pack with models in collada format, the customer can load the collada file into whatever 3d package they have that supports importing collada - does max, lightwave, maya? If that's so, that would be an awesome prospect.
#37
You will, of course, be at the mercy of how good their app's Collada loader is but, in general, I've found them to be decent. I've also had a lot of success using Ultimate Unwrap 3D to convert from various file formats to and from Collada.
02/18/2009 (2:25 am)
Andy, that is correct =)You will, of course, be at the mercy of how good their app's Collada loader is but, in general, I've found them to be decent. I've also had a lot of success using Ultimate Unwrap 3D to convert from various file formats to and from Collada.
#38
The Collada output earlier was not allowing for multiple mesh groups as separate 'objects', it now does; this was part of my begging Chris to allow me to use several different DAE 'flavors' to assemble a 'shape' inside Torque. He did just an outstanding job of anticipating and providing the functions necessary for this. I wanted to take my U3D DAE output as 'animations' and my Ms3d output for the 'shape' part of it, since it was holding the subMesh data nicely, and so on....
Again, I think this aspect of the 'Collada' read/Loader to be the 'Big' story here;
02/18/2009 (2:31 am)
U3D does a good job of converting a lot of fileTypes...in fact; there was a recent update of the FBX, DAE, and XSI plugins....The Collada output earlier was not allowing for multiple mesh groups as separate 'objects', it now does; this was part of my begging Chris to allow me to use several different DAE 'flavors' to assemble a 'shape' inside Torque. He did just an outstanding job of anticipating and providing the functions necessary for this. I wanted to take my U3D DAE output as 'animations' and my Ms3d output for the 'shape' part of it, since it was holding the subMesh data nicely, and so on....
Again, I think this aspect of the 'Collada' read/Loader to be the 'Big' story here;
expansion of the TSShapeConstructor Class
....:)
#39
02/18/2009 (4:32 am)
I couldn't help but notice the Tim Aste model in the video and demo app. How did you guys manage to get that into collada? Is there a map to collada converter or was this done manually? 
#40
Hopefully you're able to get past the limitations of Collada, because having a direct method for getting fully-functional content into T3D will be very useful.
02/18/2009 (5:19 am)
Great work! This is the kind of news I've been waiting for. Please, please, please put a lot of focus on the art pipeline. :)Hopefully you're able to get past the limitations of Collada, because having a direct method for getting fully-functional content into T3D will be very useful.
Torque 3D Owner Black Tengu