Silhouette selection via postFx for Torque3D
by Konrad Kiss · 07/13/2009 (5:43 pm) · 42 comments
This resource adds a new postFx to Torque3D that lets you have object selection with a line silhouette and an overlay. Here's an image to give you a better idea:
Before I go on with the resource, I'd like to express my thanks to those whom without I'd not been able to wrap this up now. They are:
Greg G for starting this thread.
Matt Jolly, who's made an awesome postFx, which started me on the road to figure out a lot more about shaders and postFx in T3D Beta3.
Tom Spilman for giving me glow code from Beta 3 before it was even out to be able to get started faster.
Huan Li for pointing out a change in code I couldn't find, which eventually led to this resource within a matter of hours. Also, for testing this resource!
Thanks again, guys.
One more thing, this resource does not include the method of selecting an object, only the way to render it as above. When you have your selection working, you can use this resource to make a ShapeBase to render as selected.
Alright, here we go:
0.) Don't forget to do a backup.
1.) Download the resource's files in this Torque3D only forum, and copy them into your game's source. The directory structure is the same in the zip as Torque3D's default dir structure, so you can just copy both the source and the game directory over your existing directories. No existing files should be overwritten. Don't recompile just yet.
2.) Engine source changes:
baseMatInstance.h
- Add the protected boolean mHasSelection to the BaseMatInstance class
- Also here, add the following as public methods:
matInstance.cpp
- in MatInstance::construct, set mHasSelection to false
renderPassManager.cpp
- include the new renderSelectionMgr.h
- in MeshRenderInst::clear, set mHasSelection to false
renderPassManager.h
- Add the boolean mHasSelection to the MeshRenderInst struct
sceneData.h
- Add the following to the SceneGraphData struct's BinType enum:
shapeBase.cpp
- Insert into ShapeBase.cpp and call this method when your object's (this) selection changes. Depending on your selection method, there are different ways to handle this, but at the end, you should call this to set the selection state of the mesh.
shapeBase.h
- Add the following as a public method to the ShapeBase class definition:
3.) Script changes:
core/scripts/client/renderManager.cs
- Add the following code to the end of the initRenderManager function:
Now it's time to recompile.
To turn this feature on, enter SelectionPostFx.enable() at the console, and make sure you use setSelection on a ShapeBase in view.
I still want to do dynamic coloring sometime. Right now, the shader's hardwired to do a shades-of-red silhouette and overlay. If you want to change the color, here's where you'll need to make modifications in selectionShaderP.hlsl:
That's all. Let me know if I forgot something! (It's possible, since I just extracted this from my game. I've tried to double-check all changes, but haven't yet had the time to test it in vanilla T3D Beta 3)
Thanks for following through.
--Konrad
Before I go on with the resource, I'd like to express my thanks to those whom without I'd not been able to wrap this up now. They are:
Greg G for starting this thread.
Matt Jolly, who's made an awesome postFx, which started me on the road to figure out a lot more about shaders and postFx in T3D Beta3.
Tom Spilman for giving me glow code from Beta 3 before it was even out to be able to get started faster.
Huan Li for pointing out a change in code I couldn't find, which eventually led to this resource within a matter of hours. Also, for testing this resource!
Thanks again, guys.
One more thing, this resource does not include the method of selecting an object, only the way to render it as above. When you have your selection working, you can use this resource to make a ShapeBase to render as selected.
Alright, here we go:
0.) Don't forget to do a backup.
1.) Download the resource's files in this Torque3D only forum, and copy them into your game's source. The directory structure is the same in the zip as Torque3D's default dir structure, so you can just copy both the source and the game directory over your existing directories. No existing files should be overwritten. Don't recompile just yet.
2.) Engine source changes:
baseMatInstance.h
- Add the protected boolean mHasSelection to the BaseMatInstance class
- Also here, add the following as public methods:
// >>>
bool hasSelection() { return mHasSelection; };
void setSelection(bool sel) { mHasSelection = sel; };
// <<<matInstance.cpp
- in MatInstance::construct, set mHasSelection to false
renderPassManager.cpp
- include the new renderSelectionMgr.h
- in MeshRenderInst::clear, set mHasSelection to false
renderPassManager.h
- Add the boolean mHasSelection to the MeshRenderInst struct
sceneData.h
- Add the following to the SceneGraphData struct's BinType enum:
// >>>
/// The selection render bin.
/// @RenderSelectionMgr
SelectionBin,
// <<<shapeBase.cpp
- Insert into ShapeBase.cpp and call this method when your object's (this) selection changes. Depending on your selection method, there are different ways to handle this, but at the end, you should call this to set the selection state of the mesh.
// >>>
void ShapeBase::setSelection( bool sel )
{
if (!mShapeInstance || !isClientObject())
return;
if (!mShapeInstance->ownMaterialList())
return;
TSMaterialList* pMatList = mShapeInstance->getMaterialList();
for (S32 j = 0; j < pMatList->getMaterialInstCount(); j++)
{
BaseMatInstance * bmi = pMatList->getMaterialInst(j);
bmi->setSelection(sel);
}
}
// <<<shapeBase.h
- Add the following as a public method to the ShapeBase class definition:
// >>> void setSelection( bool sel ); // <<<
3.) Script changes:
core/scripts/client/renderManager.cs
- Add the following code to the end of the initRenderManager function:
// >>>
DiffuseRenderPassManager.addManager( new RenderSelectionMgr() { renderOrder = 1.6; processAddOrder = 1.6; } );
// <<<Now it's time to recompile.
To turn this feature on, enter SelectionPostFx.enable() at the console, and make sure you use setSelection on a ShapeBase in view.
I still want to do dynamic coloring sometime. Right now, the shader's hardwired to do a shades-of-red silhouette and overlay. If you want to change the color, here's where you'll need to make modifications in selectionShaderP.hlsl:
float4 e = float4(vis, 0, 0, vis); // <-- silhouette color float4 ovr = float4(avgval, 0, 0, avgval); // <-- overlay color ovr *= 0.4; // <-- overlay "thickness" (0.4 = 40% transparency for the overlay)
That's all. Let me know if I forgot something! (It's possible, since I just extracted this from my game. I've tried to double-check all changes, but haven't yet had the time to test it in vanilla T3D Beta 3)
Thanks for following through.
--Konrad
About the author
See www.bitgap.com.
#22
I might have changed something without taking note of it in this thread, sorry about that. I have posted my entire render function that works with Beta 5 into the resource thread where the files can be downloaded.
09/10/2009 (11:53 am)
Hey Rex,I might have changed something without taking note of it in this thread, sorry about that. I have posted my entire render function that works with Beta 5 into the resource thread where the files can be downloaded.
#23
I thought it may be a network issue so I created a command on the client side that accepts an objectID and a boolean value. This function calls a severCmd function that calls the setSelection method of the object. Nothing seems to work. Yes, I did enable the PostFx before attempting.
Thanks for your help.
09/10/2009 (9:59 pm)
The updated code works but I'm having trouble with the selection. I hard coded the HasSelection to be true and it rendered everything red with an outline as expected. Once I set that back to normal I tried calling the setSlection function on an object from the console but still doesn't seem to work.I thought it may be a network issue so I created a command on the client side that accepts an objectID and a boolean value. This function calls a severCmd function that calls the setSelection method of the object. Nothing seems to work. Yes, I did enable the PostFx before attempting.
Thanks for your help.
#24
09/10/2009 (10:12 pm)
@Rex: Call setSelection on the client side. You need to take care of doing the selection, and passing that down to the client. Once there, call setSelection on every object (ShapeBasse derivative here) that had its selection state changed (selected, unselected).
#25
Right now i am just trying to test from the consol. I open the mission in the mission editor and take note of the ID of an object that has a datablock. In this case it is a door that opens. After enabling SelectionPostFx i try "3011.setSelection(true);" on the console. I added a few lines to the code so that i can see where the problem is. It is saying this is not a client object but i am at a loss as to why. What am i missing?
09/10/2009 (11:24 pm)
Still new to this. And just when i think i'm making progress, something this simple trips me up.Right now i am just trying to test from the consol. I open the mission in the mission editor and take note of the ID of an object that has a datablock. In this case it is a door that opens. After enabling SelectionPostFx i try "3011.setSelection(true);" on the console. I added a few lines to the code so that i can see where the problem is. It is saying this is not a client object but i am at a loss as to why. What am i missing?
#26
This might have been a little overwhelming, so.. I'll try to explain a bit better.
When you open the console, and you are running a local server, then whatever you enter in the console will be run server side. That's very important to note. You can however address the local client using LocalClientConnection (search for some examples in the scripts).
This local client connection - or any client connection for that matter - will not store all the objects on the client as the server, only the ones that are "scoped" to the client - the ones that are in range for the client camera. Not all the objects are scoped to the client, some remain on the server only. This depends on the purpose of an object.
This resource makes it possible for you to select any ShapeBase object on the client by flipping a flag on it by using setSelection on the object. Look at what setSelection does - it earlies out of the method if the object is not a client side object. It will not run on the server side at all.
To make it run on the client side, you need to find the selectable ShapeBase derivative object's id on the client, and set the selection flag this way:
That %client could be LocalClientConnection if you're making a single player game, or any of the connected clients. The object id will be the object's id on the client. This is probably not the same as the very same object's id on the server. This way the client can not assemble an object list found on the server - it is a simple yet effective protection.
The copy of a server object on the client is called a ghost. What you need to find is the id of the ghosted object on the client. You can ask the server what the ghost id of a server object is on a specific client. You should probably read up on that, there are topics on that all over the forums. I'm not on my dev machine, so I can't offer code examples right now, but let me know if you need more help.
09/11/2009 (12:55 am)
It must be a server object. On the client side, this object has a ghost. What you need to do is get the object's client side ghost id on the server, and resolve that id on the client which will give you the correct object. This might sound confusing, but it is a great way to protect objects on the server from a malicious client.This might have been a little overwhelming, so.. I'll try to explain a bit better.
When you open the console, and you are running a local server, then whatever you enter in the console will be run server side. That's very important to note. You can however address the local client using LocalClientConnection (search for some examples in the scripts).
This local client connection - or any client connection for that matter - will not store all the objects on the client as the server, only the ones that are "scoped" to the client - the ones that are in range for the client camera. Not all the objects are scoped to the client, some remain on the server only. This depends on the purpose of an object.
This resource makes it possible for you to select any ShapeBase object on the client by flipping a flag on it by using setSelection on the object. Look at what setSelection does - it earlies out of the method if the object is not a client side object. It will not run on the server side at all.
To make it run on the client side, you need to find the selectable ShapeBase derivative object's id on the client, and set the selection flag this way:
%client.<SimObjectId objectid>.setSelection(<bool selectionstate>);
That %client could be LocalClientConnection if you're making a single player game, or any of the connected clients. The object id will be the object's id on the client. This is probably not the same as the very same object's id on the server. This way the client can not assemble an object list found on the server - it is a simple yet effective protection.
The copy of a server object on the client is called a ghost. What you need to find is the id of the ghosted object on the client. You can ask the server what the ghost id of a server object is on a specific client. You should probably read up on that, there are topics on that all over the forums. I'm not on my dev machine, so I can't offer code examples right now, but let me know if you need more help.
#27
09/11/2009 (10:31 pm)
Ah! That helps a lot! I'm still trying to get used to how ghosting works. But I did get this resource working and it looks great! I found a function "serverToClientObject" that allowed me to get the client version of the object and highlight it. I may try changing the highlight code to take a number instead of a bool so that the state can be indicated by different shaders. But for the moment, it works. Thanks for all of your help and quick responses.
#28
Good luck with further expanding it! Passing a number instead of a bool sounds like a cool idea!
09/11/2009 (11:16 pm)
Glad it works for you! This is probably one of those resources that might be hard to get a grip on at first, but explains a lot about the engine once you do!Good luck with further expanding it! Passing a number instead of a bool sounds like a cool idea!
#29
09/23/2009 (10:51 pm)
Awesome resource! I was able to get this working with Dave Myer's object selection resource pretty fast (especially considering the first time I integrated this, I misplaced code in like five places, but it still compiled just fine, lol).
#30
09/23/2009 (11:25 pm)
Hey Ted, very cool! Good to know it works for you! :) 
#31
09/23/2009 (11:48 pm)
@Ted - Did you try this resource out with the code I sent you recently? I plan on using this in some docs as well, and it will have to work with the custom code from the tutorial I sent you. Any luck?
#32
But the changes I made in relation to this are as follows (this assumes Dave Myers's resource is used, but it shouldn't be hard to mod it):
In scripts/client/game.cs, I added a clientCmd function to do the selection. %clientObj is sent by the server, and the %set is 1 or 0 to turn it on and off:
In scripts/server/commands.cs, I changed the serverCmdSelectObject() function to resolve the client object and then send that id down to the client to get highlighted (that may bite me in the ass in testing, so it can be done with the ghost id resolution functions too):
I also created a ConsoleMethod named setSelection() in shapeBase.cpp:
For the purposes of your docs, I guess you can do the following (you'll need the ConsoleMethod for this, and I'd recommend a variable storing the object id selected for deselecting, or I could just be paranoid because I use that for a lot of other stuff):
Let me know if that works.
09/24/2009 (12:12 am)
Actually, I wound up porting Dave Myers's object selection resource over, because of the networked nature of it and it having the actual selection done on the server-side, whereas your resource seemed geared towards RTS stuff.But the changes I made in relation to this are as follows (this assumes Dave Myers's resource is used, but it shouldn't be hard to mod it):
In scripts/client/game.cs, I added a clientCmd function to do the selection. %clientObj is sent by the server, and the %set is 1 or 0 to turn it on and off:
function clientCmdSetObjSelection(%clientObj, %set)
{
if(%clientObj)
%clientObj.setSelection(%set);
}In scripts/server/commands.cs, I changed the serverCmdSelectObject() function to resolve the client object and then send that id down to the client to get highlighted (that may bite me in the ass in testing, so it can be done with the ghost id resolution functions too):
if (%scanTarg)
{
%targetObject = firstWord(%scanTarg);
%clObj = serverToClientObject(%targetObject);
commandToClient(%client, 'setObjSelection', %clObj, 1);
%client.player.tgtID = %targetObject;
.......//other stuff
}
else
{
%targetObject = %client.player.tgtID; //I store the target ID for lots of reasons, not just highlighting, but that works good too
%clObj = serverToClientObject(%targetObject);
commandToClient(%client, 'setObjSelection', %clObj, 0);
........//other stuff
}I also created a ConsoleMethod named setSelection() in shapeBase.cpp:
ConsoleMethod(ShapeBase, setSelection, void, 3, 3, "(bool) Enables/disables object selection highlighting")
{
object->setSelection(dAtob(argv[2]));
}For the purposes of your docs, I guess you can do the following (you'll need the ConsoleMethod for this, and I'd recommend a variable storing the object id selected for deselecting, or I could just be paranoid because I use that for a lot of other stuff):
// onMouseDown is called when the left mouse
// button is clicked in the scene
// %pos is the screen (pixel) coordinates of the mouse click
// %start is the world coordinates of the camera
// %ray is a vector through the viewing
// frustum corresponding to the clicked pixel
function PlayGui::onMouseDown(%this, %pos, %start, %ray)
{
%ray = VectorScale(%ray, 1000);
%end = VectorAdd(%start, %ray);
// Only care about players this time
%searchMasks = $TypeMasks::PlayerObjectType;
// Search!
%scanTarg = ContainerRayCast( %start, %end, %searchMasks );
// Get our player/actor
%ai = LocalClientConnection.player;
// If an enemy AI object was found in the scan
if( %scanTarg )
{
// Get the enemy ID
%target = firstWord(%scanTarg);
//For object selection highlighting...
%clObj = serverToClientObject(%target );
if(%clObj)
%clObj.setSelection(1);
// Don't shoot at yourself
if( %target != %ai )
{
// Cause our AI object to aim at the target
// offset (0, 0, 1) so you don't aim at the target's feet
%ai.setAimObject(%target, "0 0 1");
// Tell our AI object to fire its weapon
%ai.setImageTrigger(0, 1);
return;
}
}
// If no valid target was found, or left mouse
// clicked again on terrain, stop firing and aiming
%ai.setAimObject(0);
%ai.setImageTrigger(0, 0);
}Let me know if that works.
#33
Thanks!
RMH
09/30/2009 (9:11 pm)
Looks like a few things have changed with the release of T3D v1.0 that break this resource. I tried looking at how to fix it but I am very new to the rendering code. Any updates?Thanks!
RMH
#34
09/30/2009 (9:16 pm)
I'll start porting this and my other resources to 1.0 in a few days, so that's when I'll be posting updates.
#35
In ShapeBase::setSelection in ShapeBase.cpp change
In renderSelectionMgr.cpp
add the following include:
then replace RenderSelectionMgr::render with this:
09/30/2009 (9:31 pm)
Got it!In ShapeBase::setSelection in ShapeBase.cpp change
for (S32 j = 0; j < pMatList->getMaterialInstCount(); j++)to
for (S32 j = 0; j < pMatList->mMatInstList.size(); j++)
In renderSelectionMgr.cpp
add the following include:
#include "math/util/matrixSet.h"
then replace RenderSelectionMgr::render with this:
void RenderSelectionMgr::render( SceneState *state )
{
PROFILE_SCOPE( RenderSelectionMgr_Render );
// Don't allow non-diffuse passes.
if ( !state->isDiffusePass() )
return;
GFXDEBUGEVENT_SCOPE( RenderSelectionMgr_Render, ColorI::GREEN );
GFXTransformSaver saver;
// Restore transforms
MatrixSet &matrixSet = getParentManager()->getMatrixSet();//ADDED
matrixSet.restoreSceneViewProjection();//ADDED
// Tell the superclass we're about to render, preserve contents
const bool isRenderingToTarget = _onPreRender( state );
// Clear all the buffers to black.
GFX->clear( GFXClearTarget, ColorI::BLACK, 1.0f, 0);
// init loop data
SceneGraphData sgData;
U32 binSize = mElementList.size();
for( U32 j=0; j<binSize; )
{
MeshRenderInst *ri = static_cast<MeshRenderInst*>(mElementList[j].inst);
setupSGData( ri, sgData );
sgData.binType = SceneGraphData::SelectionBin;
BaseMatInstance *mat = ri->matInst;
U32 matListEnd = j;
while( mat->setupPass( state, sgData ) )
{
U32 a;
for( a=j; a<binSize; a++ )
{
MeshRenderInst *passRI = static_cast<MeshRenderInst*>(mElementList[a].inst);
if (newPassNeeded(mat, passRI))
break;
//mat->setTransforms(*passRI->objectToWorld,
// *passRI->worldToCamera, *passRI->projection);**REMOVED
matrixSet.setWorld(*passRI->objectToWorld);//ADDED
matrixSet.setView(*passRI->worldToCamera);//ADDED
matrixSet.setProjection(*passRI->projection);//ADDED
mat->setTransforms(matrixSet, state); //ADDED
//mat->setEyePosition(*passRI->objectToWorld, state->getCameraPosition()); **REMOVED
mat->setBuffers(passRI->vertBuff, passRI->primBuff);
if ( passRI->prim )
GFX->drawPrimitive( *passRI->prim );
else
GFX->drawPrimitive( passRI->primBuffIndex );
}
matListEnd = a;
}
// force increment if none happened, otherwise go to end of batch
j = ( j == matListEnd ) ? j+1 : matListEnd;
}
// Finish up.
if ( isRenderingToTarget )
_onPostRender();
}
#36
09/30/2009 (9:33 pm)
Awesome, thanks Rex!
#37
10/01/2009 (11:29 pm)
Ok, so one minor issue remains. The highlighting is no longer occluded by foreground objects. Any idea?
#38
Please let me know if that helps.
10/29/2009 (4:23 am)
@Rex: You could try adding the following line before setBuffers in your render function:mat->setSceneInfo(state, sgData);
Please let me know if that helps.
#39
11/06/2009 (3:08 am)
That didn't help. I've also noticed that this issue is present under Advanced Lighting but it works as expected under Basic Lighting. Is there something different with the order that things are processed under each type of lighting? I haven't been able to find anything but most of that gets over my head quickly!
#40
I haven't ported this part yet, will only do that about a week from now. If you still can't find the problem, take a look at renderGlowMgr, it's very similar to this manager, maybe you'll find what's missing.
11/06/2009 (3:17 am)
I didn't see anything that should cause this, though.. you could try to change that renderOrder and processAddOrder to 1.55 each in renderManager.cs so it doesn't conflict with some new stuff.I haven't ported this part yet, will only do that about a week from now. If you still can't find the problem, take a look at renderGlowMgr, it's very similar to this manager, maybe you'll find what's missing.
Torque 3D Owner Rex Hiebert
Error 1 error C2039: 'dynamicLight' : is not a member of 'MeshRenderInst'
the error was in the function:
void RenderSelectionMgr::render( SceneState *state )
the line is:
if(ri->dynamicLight)