Art pipeline progress update

What a strange week.

Anyway, yes, progress on the art pipeline I talked about in a previous blog entry.

So, the basic idea so far, is to load a bunch of data from whatever DCC (art) package you choose in a format of your own choosing (as long as its supported) and then plop it out into DTS in as easy a manner as possible (i.e. without any effort on your part).

So, you'll remember my diagram from the last post here

Anyway, if you look at the diagram, it has in the right hand side the "scenegraph". A scenegraph is basically a simple tree structure called a "Directed Acyclic Graph" (DAG for short). Basically almost every 3D application out there and for that part, almost all game engines (including torque sort-of) use a scene graph to store the world data that is used for rendering.

The power of a scenegraph comes into its own when you go through the render loop and are looking at the various transforms. Basically, the one-way heirarchy of a scenegraph maps brilliantly onto how the 3d hardware wants to see its information represented.

Or at least, thats the theory. In practice its not quite that clean cut, because there are plenty of other issues beyond the rendering that dont really work with a simple one-way graph. For instance when you have multiple dependancies on position data.

Anyhoo, getting back to the point of progress made.

So far this week, I've thrashed through about a dozen different scenegraph api's looking to see if any of them was simple and generic enough to use as a container for data thats read in from the various 3D model formats, ready to be massaged and fiddled with prior to export to dts/x format.

Unfortunately, I came to the conclusion that its easier and faster to just write my own little dumb-as-a-brick scenegraph API. Mainly because its only going to store information, not do anything tricky and partially because even the thought of the dependancies that most of these open source scenegraphs impose on you scares me.

So how does a scene graph work?

Well, the basic idea, is that you have a basic "node" class. This is the main method of structuring the data. The node is essentially an abstract base class (I decided not to make it pure because I might need to just throw a node into a graph for no particular reason later on). Each node has 1 parent and N child nodes (N being any number > 0).

So with that basic structure, we can build a graph. But before doing so, we need some more useful nodes!

The nodes implemented so far are:

MeshNode - holds basic vertex data (i.e. the vertices and indicies of a model).
UVSetNode - holds UV data for the same set of vertices
MaterialNode - holds basic surface information (material attributes and surface description)
TransformNode - holds a transform to apply to subsequent nodes
LODNode - basically a switch node, each LOD node will normally have N number of mesh nodes, at varying levels of detail

So the basic nodes and scenegraph structure is in place, with the appropriate "get/set" and accessor functions for adding/removing nodes in a scene in the right place etc.

So what next?

Well, the next step, is to write a recursive reader that takes the appropriate file format (in the first instance, it will be the FBX format) and constructs a "Scene" from the file. All this basically will do, is use the FBX SDK api to load its own internal scenegraph representation of the file (funnily enough, a lot of 3d apps store thier info in a scene-graph like format) and then simply recursively traverse the FBX scene throwing the appropriate data into my own scene-graph nodes.

Now, the more switched on of you will ask "why do you have your own scenegraph when FBX already has one?". Good question! Of course, the answer is to reduce the dependancy on a proprietary API and to allow us to handle ANY format of data. The internal scenegraph is much simpler than the FBX version because the FBX by its nature is intended for a far more complex set of requirements. Plus, we can more easily support other file formats if we can somehow manage to parse them into our own scenegraph, because once they are in that format, writing a scene->dts traversal will be soooo much simpler.

So, next week will mostly be loading and populating scene's from FBX files. Dumping out the various data to the console to make sure it makes sense. Maybe comparing that to some kind of FBX text output on test scenes. Perhaps (shock horror) writing some unit tests to make sure we can catch some of the more insane things that tend to go into data files and often catch the unwary app off guard (I'm sure somewhere there will be a file with a gazillion one polygon mesh's).

It may be that the unit tests will end up as an "ArtComplaint" tool, where you can ship this to your artists and have it whine and moan at them automatically for having rotten or poorly formatted art. Hell, I might even incorporate a text to speech engine just for giggles :)

Once the data is in the scenegraph. The next step is of course, to write it out to dts. Yay, so we will have a method of transforming mesh data into dts. Not exactly stellar progress, but its a stepping stone.

So no screenshots this week. But I promise to show you next week a screen dump of some text output telling you that the internal scenegraph is indeed populated with lovely vertex, material and uv data!!

PS: For the keen eyed amongst you, notice there is a UVSet node? Imagine having MULTIPLE UVSet's associated with a given mesh??? oh yes..