Yet another Skinned-Sorting w/Misc Problems Thread

#1 Bah, :P Go figure, the bones really were the problems. I happened upon some documentation by Logan Foster, stating that irregular meshes most likely will cause sorting errors. So, I remedied the number suffix problem, thus fixing the sort errors as well.

I still think transparency sorting is too finicky in Torque though. I'll leave my post untouched in case someone else might learn something from it (somehow)

#2 I'll start with the bone becoming meshes issue as this one is a simple fix; Don't name your bones with numbers at the end as the exporter will think these are meshes. As simple as that. (Just using the rename tool in max to add a letter or two to the end of the bone names should work as a simple fix).


Then... the transparency thing. Hm. Well, this have been up for discussion before and the only answer that I've seen is; sorry, it's simply not going to work. With static meshes you can usually work around it to make it work but with skinned meshes it's simply impossible with the current implementation.

What we did in Minions of Mirth to "fix" this was to use alpha masking rather than alpha blending (using an 1 bit alpha instead of a 8 bit one. That is, you only have on or off in the alpha, no in between). I'm pretty sure this required some code changes though but I'm "just the artist" so I wouldn't really know. =)

#3 Nice Horse! Stick a Viking on there and you'll be set!

#4 Hey Blikstad, yeah, the numbers at the end of the bone names eventually dawned on me :P I don't think the alpha hack is an option for me, since I'm "just the artist" also ;)

It turned out that after a little bit of switching around of bone parents/children, etc., the sort order went way out of wack again, and I was back to square one. It seems that renaming the bones had nothing to do with the sorting problem. I believe that the irregular mesh statement by Foster (no offense, Foster) is a complete myth, or at least a myth in relation to this sorting dilemma.

I found out that the sort order that each mesh is rendered is related to the arrangement order within 3dsmax. I had not read of this at all anywhere, and I found the solution that allows me to define which mesh gets drawn first in the sort order within 3dsmax, without needing a hack or anything. Doing this makes it so that I can get past any sort rendering error for any skinned or static dts shape I make, including my horse ;) I'm not entirely sure if this will work on very complex shapes (with 6-7 each of trans and opaque mesh objects), but I wouldn't be surprised if an artist could make such a complex shape's sorting much more pleasant to look at through this method.

Did anyone come across this solution? If no one or not many know about it, I'll go into more detail in tutorial-form, to show how to conquer the evil sort dilemma that's been plaguing many artists within 3dsmax. Hopefully I'm not adding hype to something that might be common knowledge.

Todd Pickens, thanks :) It'll be the size of a child's pony, so it'll look kind of funny to have a barbarian viking riding it ;)

#5 Feel free to post the solution. I'm listening. =)

#6 Alright, may take an hour or so, I still don't quite understand how the sort ordering works, I just know how to re-arrange the order until it finally looks alright. Once I either understand it or give up trying to, I'll start postin' ;)

#7 Manually Specifying the Sort Order of Static or Skinned Transparent and Opaque Meshes

First of all, I believe this can probably be done with maya, lightwave, etc, except using different commands/interfaces. However, this "tutorial" pertains to 3dsmax only.

Now, here's an explanation of how the workings between 3dsmax, the exporter, and the DTS relates to the Sort Order:

1. Firstly, the arrangement of the mesh objects within 3dsmax directly affects the order that mesh objects are rendered in the Torque Game Engine. Or more exactly, it directly affects the output of the exporter that tells the dts file which gets rendered first and last.

2. The fewer and simpler the actual "Elements" (separate meshes), the easier it is to fix the sort order to your liking, and the better the chance you'll get the quality you want.

3. Low-down on what exactly "sorting" does when in torque:

A. The first mesh specified to be rendered will be rendered first.
B. The next mesh specified will be rendered in front of the first mesh.
C. And the next will be rendered in front of the previously specified one, and so on.
D. As long as all of the translucent meshes are specified after all of the solid/opaque, all of the translucent meshes will be properly rendered behind and in front of the opaque meshes.

For now (since I couldn't figure it out), it's best to assume that the exporter uses a fixed random seed for sorting between the arrangements in max and dts. In truth, I think it's related to converting between the max and dts matrices, or something like that, but it's too much for my mind to unravel at the moment :P

Though seemingly random, the order seems to almost reverse when going from max to dts. Here is a series of combinations. The top row is the combination in 3dsmax, the bottom is the end result in torque:

3 Opaque (DQO) 3 Translucent (#%&) meshes
DQO#%& D#Q%O& #%&DQO #D%O&Q #&D%QO
%O#DQ& O%#DQ& OQD#%& &#D%QO OQ%D#&

2 Opaque 2 Translucent
D%O# DO#% %D#O %#OD OD%# O#D% #%DO #O%D
DO%# DO#% DO%# OD%# OD%# OD#% DO%# DO%#

1 Opaque 3 Translucent
D%#& D&%# D#&% #%&D #%D& #D&%
D%#& D&%# D#&% D%&# #%D& D#&%

3 Opaque 1 Translucent
DQ%O DOQ% DO%Q QDO% QD%O Q%DO %DOQ %ODQ %OQD
DQO% DOQ% DOQ% QDO% QDO% QDO% DOQ% ODQ% OQD%

This enforces the rule that you should place all of the Translucent meshes before all of the Opaque meshes, so that there's one section of Trans to the left of the Opaque section.

Indeed, it may be best to view translucent meshes and opaque meshes as two separate sections, as it could make sorting a little easier.
_____________

Ok, let's skip on to exactly what to do in 3dsmax and Torque Showtool Pro. We'll use the 3 Opaque, 3 Translucent meshes of a horse as an example.

1. After everything is modeled, rigged, mounted, with proper hierarchy so that it can load in TGE and TSP, go to your schematic view, and make sure all your meshes are parented to the same node or object. Here is the hierarchy for the horse; in this case, the mesh objects are parented to the scene root automatically:

tail2, chain2, and mane2 are the translucent meshes. saddle2, eyes2, and body2 are the opaque meshes

2.I've already done this, but you need to select all of the meshes and hit the Arrange Selected button in the top menu, or use the context menu. This arranges it in the order that it'll be exported to DTS.

What we need to do is go ahead and arrange all of the translucent meshes so that they are in front of the opaque meshes. Now, there may be an easier way to re-arrange the mesh orders, but I am not very proficient with 3dsmax, so I'll explain the way I do it.

3. The concept: The ordering works where if you take a mesh out of the same hierarchy and re-insert it, the mesh will be moved to the back of the line. The fastest way I know how to do this is by selecting one or more that you want to send to the back, and for those who...A)Have the meshes as children to a bone or node, simply press the unlink selected button or use the context menu. Then link the meshes back to the bone or node, select all of the meshes, and press the Arrange Selected button...B)Have the meshes as children to the scene root (basically, by themselves as in the picture above), connect or link those selected meshes to an unselected mesh, then unlink the ones you linked, select all of them, and press the Arrange Selected button (depending on your efficiency, you may not have to Arrange Selected).

4.Obviously, since you can only move the meshes to the back of the line rather than in specific areas, you'll have to work out the puzzle of arranging it how you want it.

Below I'll show the steps I took to put all the translucent objects in the front.

I select saddle2 and body2 together:

I connect the two meshes to one of the others--Before:
After:

I unlink the two linked meshes, they fall to the back:

5. Now, you may have separated the two sections, keeping the trans on the left and the opaque on the right, but that doesn't mean they're in the order you want it to be. At this point and time, go ahead and export the dts file and load it up in Torque Showtool Pro. To find out what was sorted before what, press the shape properties button on the left menu bar. This time the order is from top to bottom, where the first "layer" is rendered in the back, up to where the last "layer" is in front. More explained below:
Refer to my low-down on how the sort rendering works. The first two, saddle and eyes, are opaque, and there isn't really any noticeable difference in what order opaque meshes are rendered unless coupled with translucent meshes. Chain is where you begin to pay attention. The chain mesh is rendered ON TOP of the eyes in any perspective. Next, Body is rendered ON TOP of the chain mesh in every perspective. Since Chain is translucent, this is bad because the ordering makes a difference here; with the Body mesh covering the chain, you'll never see the chain wherever the body is aligned physically behind it from the user's perspective. Instead, it'll look like the body is in front of the chain, should any part of the chain stick out. The chain meshes are actually quads that act as links to the head harness, and are quite close to the body, so it's not likely that anything will stick out. Therefore, in this instance, this sort order is BAD :P

Now, next the Tail mesh becomes rendered ON TOP of the Body mesh. Like with the body on top of the chain, the tail will end up rendering over all the opaque meshes behind it, and will be properly overlapped by the opaque meshes behind it when need be. (don't ask me how it works, refer to the explanation on sorting and z-buffers, etc at tdn.garagegames.com, if anything). This is fine, and is what we want, and I won't cover how two translucent meshes will overlap each other (mane and tail) until after we fix the sort. Below's an example of what the horse looks like so far. It all looks fine except that the chains are being blocked by the body (don't mind the textures :P) :

#8 6. So, now what? Well, we go back to 3dsmax and re-arrange the meshes within the translucent section, testing the combinations out until you find a winner. I haven't worked with this method enough to know for sure, but I don't think there will be many instances whatsoever where you have to mix both the translucent and opaque sections, or have the translucent section after the opaque section.

In the end, I came up with this combination that gives me the best results I could expect for such a complex model:
Now, as you can see, all three translucent meshes are rendered last, on top of the solids. Tail gets rendered first, then chain second and will overlap tail ANYTIME the two align, even if the tail is physically in front of the chain when aligned to the user's perspective. Mane gets rendered last, and will overlap both chain and tail ANYTIME they align. Here's a shot of what I mean:

What you're looking at is the mane overlapping the tail where it can be viewed from under and behind the horse's stomach. When animated, if the tail swished to its right side, and the user's perspective had the swished part and mane aligned, the user would be seeing the mane 100% in front of the tail.

Therefore, it's best to set the sort order based on where the camera will be located and facing most of the time, should you find yourself in such a delicate situation. This horse will be viewed mostly from the side, so it should avoid some embarrassment. Simpler animated models and static meshes won't be as problematic as this one, however.

That's it, take from this what you will. I hope at least a few new users might find some answers in this thread.

One note:
Double-sided mesh objects with transparencies must have their normals facing the direction they are folded.

For meshes that have multiple folds, such as a flag with a hole in it (the hole being the transparent area), you're out of luck when it comes to this method :P

Oh yeah, this is more a long post than a tutorial, and I'm too lazy to proofread, so if I left off somewhere or skipped a step, do tell :P

#9 Great write up Raxx. This would be great to see on TDN if you have the time (or feel like handing it off to someone else).

#10 2 sided materials SHOULDN"T be used. Essentially all the DTS exporter does is make a copy and reverse the normals then merges the verts. This is bad if you use TLK. 2 sided materials do not get lit with vert normals at 0. Solution make your own copy and offset the verts slightly from one side to next.

Thanks Raxx
Matt

#11 Raxx nice method and explaination. You can also group and ungroup nodes to achieve the same affect.

For those of you who are unsure what is going on in max, here's the skinny. Max (and most other 3D applciations) have an array that holds all the nodes that are in the current scene that you are viewing. When a new node is added to your scene a new value is added to the end of the array which points to that node(note: if you want to see this for yourself use the objects command in maxscript). When you start to group, use the box trick or link nodes to other nodes for example (to name but a few), Max will begin to push and pop the objects array around to match what is going on.

Examples:

Lets say you have three nodes: Box01 , Sphere01, and Dummy, as it is right now in the objects array Box01 is 1, Sphere01 is 2 and Dummy is 3 (max array values start at 1 btw).

If we were to group all these objects together, we would affect our objects array as follows:
Group would be 1, Box01 would be 2, Sphere01 would be 3 and Dummy is 4. This is because all of the nodes we previously had are now held within the group node and this is how Max represents this internally.

If we were to remove Sphere01 from the group we would get the following:
Group would be 1, Box01 would be 2, Dummy would be 3 and Sphere01 would be 4. Why is Sphere01 4th? Easy, because it was removed from the group and as such it gets pushed out again to the end of the list.

So what happens with linking and Hierarchy? Well it actually works exactly the same as it does with grouping, children are placed after their parents in the scene array.

What about the Box trick? Same sort of idea. With the box trick you are actually creating a new node, which goes to the end of the scene objects array list and then when you attach geometry to that box you just created, you are in affect removing the object from the scene list and pushing all its data into a new array entry.


So why does this affect how the DTS file is generated? Well this should be elementary by now. The DTS file is built via what the exporter sees from Max's internal objects array, it counts through the scene nodes from first to last and adds them to the DTS file. This is why what Raxx has described above works with handling the drawing order. In fact by this same logic, you can also use it to affect the drawing order of Elements and Faces on nodes that you export.

If you are at all half decent with max script you could manually do this yourself without needing to group, link, box trick or anything else just by affecting the objects array.

So how does this affect Torque?

Well when Torque reads the DTS file it follows a pretty simple set of logic (I am sure someone could give a more in-depth description but here is the skinny).

For loop through all the Nodes starting at the first
{
For loop through all the elements on the nodes starting at the first element it finds on the node
{
For loop through all the faces on the element, starting at the first, and draw them.
{}
}
}

As such now you can see how you have affected a long long list of changes with how things are rendered simply by changing the objects array order within your 3D application.

Anyways I hope that helps add some technical explaination to what Raxx has described above.

Logan

#12 Thanks for your input guys, I was hoping others would follow up on what I wrote and post more information or improvements on my method. If I have first dibs, I'd like to make a tdn article that combines everything posted in this forum (giving credit to the informants, of course) once this topic dies out and as much as possible has been revealed.

Matthew, I've tried the cloning-flipping-attaching-merging deal with making my own 2-sided meshes only before this method, but I'll follow what you say and give it a shot with the horse, while no doubt as you say there will be lighting benefits, perhaps there'll be a difference with the sort rendering...though I doubt it :P

Logan, thanks for the insight, I wish I had some knowledge of max script, perhaps eventually a script could be made public that helps automate the process.

Oh, I forgot to mention this, but before I found that linking affects the hierarchy, I used the cloning of meshes instead. This works the same, except is slower to work with compared to grouping or linking.

Also, regarding individual Element ordering, which I did notice when studying the method, I suppose detaching and attaching again would move the selected element(s) to the back of the list within the mesh node/object? The same for selected polygons, perhaps.

I'll work out a tutorial in my free time, that better shows what's happening and how the sorting works with nodes/mesh objects, elements, and polygon, along with everything else explained in the horse tutorial and by others, then post it in tdn. May be a few days though.

#13 On the subject of 2 sided materials, Somtimes the inner duplicated faces need to be a seperate object. Of course the Inner should come before the outer in either heirarchy or scene order.

Logan mentioned a script that might make the job easier. Good Idea I think a Scene order tool in the form of the scematic veiw might be a good idea.
Let ya know.
Matt

#14 I don't know if anyone else has undertaken the task, but I've started working on the script, hopefully I'll have it done within a few days...

#15 Sorry for not commenting earlier, been busy. (and a small disclaimer; I've only had time to quickly skim through all the replies so if I missed something important, sorry. I'll read it all eventually =)

Sounds like a good, or in anyway decent, solution. As you mentioned it would still work best from some angles but just placing the transparent objects last should help somewhat. It was more or less what I expected the solution to be but I hadn't really considered HOW to change the order in the internal max object arrays.

I was actually planning to do some work on a script for this, but if you already started I'll just leave it to you. I have too much stuff to do anyway.
If you need any help just let me know though, maxscript is my thing. Use it every day =)

I'm wondering if it's possible to access the object arrays and reorder them directly with maxscript though or if you have to resort to trickery such as grouping/linking/whatever?

#16 From what I can see in the maxscript documentation, you can directly re-arrange the order of the arrays. This is my first maxscript, and I'm loving and hating every second of it. The user interface will be relatively simple, which will allow the user to re-arrange selected nodes/mesh objects, elements, and polygons, and if I can figure it out, there'll be a list showing how it'll be arranged after exporting to DTS as well.

#17 Alright. Sounds good!
As I said, just let me know if you need any help with anything. Been using maxscript pretty much since it first became available. =)

#18 I know this may be a little unrelated, but none of the links for the max6_7tools.zip file on garagegames site seem to work. Would anyone mind hosting the file for a day with a link so I could grab it, or linking to a download that currently works? Would be much appreciated... Thanks!

#19 I know this may be a little unrelated, but none of the links for the max6_7tools.zip file on garagegames site seem to work. Would anyone mind hosting the file for a day with a link so I could grab it, or linking to a download that currently works? Would be much appreciated... Thanks!

#20 I discovered this before, and after looking through tdn.garagegames.com, I found that there was one compiled by The Atomizer available to download off their website http://www.theatomizer.com/