Silhouette selection via postFx for Torque3D

This resource adds a new postFx to Torque3D that lets you have object selection with a line silhouette and an overlay. Here's an image to give you a better idea:



Before I go on with the resource, I'd like to express my thanks to those whom without I'd not been able to wrap this up now. They are:

Greg G for starting this thread.
Matt Jolly, who's made an awesome postFx, which started me on the road to figure out a lot more about shaders and postFx in T3D Beta3.
Tom Spilman for giving me glow code from Beta 3 before it was even out to be able to get started faster.
Huan Li for pointing out a change in code I couldn't find, which eventually led to this resource within a matter of hours. Also, for testing this resource!

Thanks again, guys.

One more thing, this resource does not include the method of selecting an object, only the way to render it as above. When you have your selection working, you can use this resource to make a ShapeBase to render as selected.

Alright, here we go:

0.) Don't forget to do a backup.

1.) Download the resource's files in this Torque3D only forum, and copy them into your game's source. The directory structure is the same in the zip as Torque3D's default dir structure, so you can just copy both the source and the game directory over your existing directories. No existing files should be overwritten. Don't recompile just yet.

2.) Engine source changes:

baseMatInstance.h

- Add the protected boolean mHasSelection to the BaseMatInstance class

- Also here, add the following as public methods:

// >>>
   bool hasSelection() { return mHasSelection; };
   void setSelection(bool sel) { mHasSelection = sel; };
   // <<<

matInstance.cpp

- in MatInstance::construct, set mHasSelection to false


renderPassManager.cpp

- include the new renderSelectionMgr.h

- in MeshRenderInst::clear, set mHasSelection to false


renderPassManager.h

- Add the boolean mHasSelection to the MeshRenderInst struct


sceneData.h

- Add the following to the SceneGraphData struct's BinType enum:

// >>>
      /// The selection render bin.
      /// @RenderSelectionMgr
      SelectionBin,
      // <<<

shapeBase.cpp

- Insert into ShapeBase.cpp and call this method when your object's (this) selection changes. Depending on your selection method, there are different ways to handle this, but at the end, you should call this to set the selection state of the mesh.

// >>>
void ShapeBase::setSelection( bool sel )
{
   if (!mShapeInstance || !isClientObject())
      return;

   if (!mShapeInstance->ownMaterialList())
      return;

   TSMaterialList* pMatList = mShapeInstance->getMaterialList();
   for (S32 j = 0; j < pMatList->getMaterialInstCount(); j++) 
   {
      BaseMatInstance * bmi = pMatList->getMaterialInst(j);
      bmi->setSelection(sel);
   }
}
// <<<

shapeBase.h

- Add the following as a public method to the ShapeBase class definition:

// >>>
   void setSelection( bool sel );
   // <<<

3.) Script changes:

core/scripts/client/renderManager.cs

- Add the following code to the end of the initRenderManager function:

// >>>
   DiffuseRenderPassManager.addManager( new RenderSelectionMgr() { renderOrder = 1.6; processAddOrder = 1.6; } );
   // <<<

Now it's time to recompile.

To turn this feature on, enter SelectionPostFx.enable() at the console, and make sure you use setSelection on a ShapeBase in view.

I still want to do dynamic coloring sometime. Right now, the shader's hardwired to do a shades-of-red silhouette and overlay. If you want to change the color, here's where you'll need to make modifications in selectionShaderP.hlsl:

float4 e = float4(vis, 0, 0, vis); // <-- silhouette color
   float4 ovr = float4(avgval, 0, 0, avgval); // <-- overlay color

   ovr *= 0.4; // <-- overlay "thickness" (0.4 = 40% transparency for the overlay)

That's all. Let me know if I forgot something! (It's possible, since I just extracted this from my game. I've tried to double-check all changes, but haven't yet had the time to test it in vanilla T3D Beta 3)

Thanks for following through.

--Konrad