1.4's DynamicTexture and GUIs
by Tom Spilman · in Torque Game Engine Advanced · 05/04/2005 (2:07 am) · 101 replies
I'm currently looking to add rendering and interacting with a GuiControl mapped on to a shape or interior surface. First i'm trying to figure out what the best way to place this functionality into TSE. I looked at DynamicTexture from 1.4, but it's just a class and really doesn't elaborate on how it can/should be used in the overal flow of the engine.
Ideally the workflow would be like so:
1. Artist makes some *thing* and adds a dummy texture with a special name on it for the surface where the gui will be mapped.
2. The material is added which somehow marks one of it's textures as being a "gui texture" and the name of the GuiControl to render to it.
3. Thru a raycast i should be able to find the material hit and if it has a GuiControl associated with it. If so it activates it for interactivity.
Step 1 is easy. At the moment i'm mostly concerned with step 2.
Ideally this functionality would go into the Material class itself and not CustomMaterial or some other class derived from Material. This would allow a so called GuiTexture to be assigned to any stage/texture property and give the artist the most flexibility with effects overlaid on the gui texture.
I was thinking of something along the lines of what CustomMaterial does for it's special textures like $cubemap, $lightmap, $fog, $backbuff, etc. You could specify a $gui for one of the texture stages and have a gui property on the material datablock which is the guicontrol to assign to that GuiTexture.
Still... who updates the textures and GuiControls? I'm just starting to dig into this, but has anyone have any thoughts on how to integrate such a system into TSE?
Ideally the workflow would be like so:
1. Artist makes some *thing* and adds a dummy texture with a special name on it for the surface where the gui will be mapped.
2. The material is added which somehow marks one of it's textures as being a "gui texture" and the name of the GuiControl to render to it.
3. Thru a raycast i should be able to find the material hit and if it has a GuiControl associated with it. If so it activates it for interactivity.
Step 1 is easy. At the moment i'm mostly concerned with step 2.
Ideally this functionality would go into the Material class itself and not CustomMaterial or some other class derived from Material. This would allow a so called GuiTexture to be assigned to any stage/texture property and give the artist the most flexibility with effects overlaid on the gui texture.
I was thinking of something along the lines of what CustomMaterial does for it's special textures like $cubemap, $lightmap, $fog, $backbuff, etc. You could specify a $gui for one of the texture stages and have a gui property on the material datablock which is the guicontrol to assign to that GuiTexture.
Still... who updates the textures and GuiControls? I'm just starting to dig into this, but has anyone have any thoughts on how to integrate such a system into TSE?
About the author
Tom is a programmer and co-owner of Sickhead Games, LLC.
#82
09/12/2006 (9:23 am)
Anyone had any luck getting this ported to MS4? It crashes on startup for me with no debugging information. Will post a solution when I find one, figured I'd ask first though. 
#83
Here's what I have:
Now, I know %results should contain the material name and UV coordinates. However, it comes back as "" every time. Any suggestions or examples on how to get the correct results?
Also, to perform a mouse click on a button in the GUI, woud I call GuiTextureCanvas::doMouseClick passing in the UV coodinates?
11/11/2006 (2:46 pm)
Hello all. I've implemented the GuiTextureCanvas, and currently have a GUI rendering on an object without a problem. However, I'm stumped on how to get the mouse to interact with the GUI. I've read the resource and the forum thread, but I was wondering if I could get a solid example, or feedback on what I'm doing wrong.Here's what I have:
function customEditor::on3DMouseDown(%this)
{
// Determine cursor 3D Pos and Vec
%mouseVec = %this.getMouse3DVec();
%cameraPoint = %this.getMouse3DPos();
// Tell the server to perform the ray cast
commandToServer('GuiRayCast', %mouseVec, %cameraPoint);
}
function serverCmdGuiRayCast(%client, %mouseVec, %cameraPoint)
{
// Create the ray start and end points
%selectRange = 50;
%mouseScaled = VectorScale(%mouseVec, %selectRange);
%rangeEnd = VectorAdd(%cameraPoint, %mouseScaled);
// There is only one object containing the GUI,
// so the first ray cast is not needed.
%id = objWithGui.getId();
%results = GuiTextureCanvas::castRay(%id, %cameraPoint, %rangeEnd);
echo("Meter check on: " @ %results);
}Now, I know %results should contain the material name and UV coordinates. However, it comes back as "" every time. Any suggestions or examples on how to get the correct results?
Also, to perform a mouse click on a button in the GUI, woud I call GuiTextureCanvas::doMouseClick passing in the UV coodinates?
#84
Your object must be a StaticShape, not to be confused with Static_Shape. So you will need something like this to make it so.
Next the code to handle it. In the clients default.bind.cs i have;
Now again back in the clients game.cs.
and finally your materials.cs for the shape
This works for me, but there is some iffyness with the eye position, looking into that next.
03/07/2007 (8:30 pm)
I have figured out how to use this correctly.Your object must be a StaticShape, not to be confused with Static_Shape. So you will need something like this to make it so.
// consoles
datablock TSShapeConstructor(con_BasicConsole)
{
baseShape = "~/data/shapes/monitor/mon.dts";
};
datablock StaticShapeData(BasicConsole)
{
category = "Console";
className = "c_BasicConsole";
shapeFile = "~/data/shapes/monitor/mon.dts";
}; Now your object will be within the Console Category under Shapes in the World Builder.Next the code to handle it. In the clients default.bind.cs i have;
moveMap.bindCmd(keyboard, "e", "commandToServer('UseStuff');", "");In the servers commands.csfunction serverCmdUseStuff(%client)
{
// Graciously stolen from the vehicle mount resource.
%selectRange = 3;
%searchMasks = $TypeMasks::StaticShapeObjectType | $TypeMasks::StaticObjectType;
%pos = %client.player.getEyePoint();
// Start with the shape's eye vector...
%eye = %client.player.getEyeVector();
%eye = vectorNormalize(%eye);
%vec = vectorScale(%eye, %selectRange);
%end = vectorAdd(%vec, %pos);
%scanTarg = ContainerRayCast(%pos, %end, %searchMasks);
if (%scanTarg)
{
%targetObject = firstWord(%scanTarg);
%results = GuiTextureCanvas::castRay(%targetObject, %pos, %end);
if (%results)
commandToClient(%client, 'consoleDoMouse',%results);
}
}Now again back in the clients game.cs.
function clientCmdconsoleDoMouse(%args)
{
%mat = firstWord(%args);
%x = getWord(%args, 1);
%y = getWord(%args, 2);
%mat.guiTextureCanvas.doMouseClick(%x, %y);
}and finally your materials.cs for the shape
new GuiTextureCanvas( MonitorCanvas )
{
guiControl = "BaseConsole";
};
new Material(MONITORMat)
{
guiTextureCanvas = "MonitorCanvas";
baseTex[0] = "$gui";
emissive[0] = true;
mapTo = "MONITOR";
};This works for me, but there is some iffyness with the eye position, looking into that next.
#85
If I shoot a bullet, all of a sudden the iffyness goes away completely, and the GuiCrosshair lines up perfectly with where the button clicks should be going.
Something about shooting a projectile is 'initializing' something so that the coordinates start working correctly.
A little bit of instrumentation on the serverCmdUseStuff function shown above shows that after shooting a projectile, the %client.player.getEyePoint() function starts returning a different eye position.
If anyone has any knowledge of where in the code something would be changing the eye position when a projectile is fired the first time, it would be helpful to have some places to go looking.
03/10/2007 (2:01 pm)
This is an excellent resource. Following all the posts along, I have this working as well. I also get the iffyness in the eye position, but...If I shoot a bullet, all of a sudden the iffyness goes away completely, and the GuiCrosshair lines up perfectly with where the button clicks should be going.
Something about shooting a projectile is 'initializing' something so that the coordinates start working correctly.
A little bit of instrumentation on the serverCmdUseStuff function shown above shows that after shooting a projectile, the %client.player.getEyePoint() function starts returning a different eye position.
If anyone has any knowledge of where in the code something would be changing the eye position when a projectile is fired the first time, it would be helpful to have some places to go looking.
#86
03/10/2007 (9:15 pm)
Weapons can correct aim point to compensate to hit where the player is looking. It might be something to do with this that is causing this behavior.
#87
I had a suspicion that maybe it has something to do with an animation being triggered in relation to weapon firing, but I got sidetracked trying to get it working on interiors. I'm almost there... I have the material but not the UV coords.
Anyone have any hints on how I would go about getting the UV coords from near the end of the Interior::castRay_r function in interiorCollision.cpp ?
I've overloaded castRay_r to have an alternate version that fills in a TriRayInfo structure in the same way that the resource provided an alternate raycast for a mesh.
I'm getting the correct material ID, but I'm not sure how to go about getting at the UV coordinates for the interior surface although I know which surface it is.
I suspect that if I could get a triangle out of the surface somehow and the UV coords that correspond to it's vertices, I could use the getBarryCentricCoord function that was part of the resource to get the correct UV coords.
For anyone else trying to get this working, GuiControls constructor sets it's bounds to 640x480, so if you create your GUI control at 512x512 to satisfy the power of two requirements for a texture, that will also throw off the hit point calculation. I just set the bounds to 512x512 in the constructor for GuiTextureCanvas, but that's not very satisfying or flexible. Ideally, GuiTextureCanvas sizes its texture to match the size of the GUI you hook up to it, but for now I'm happy that i's working as well as it is.
03/10/2007 (9:49 pm)
Thanks Ben!I had a suspicion that maybe it has something to do with an animation being triggered in relation to weapon firing, but I got sidetracked trying to get it working on interiors. I'm almost there... I have the material but not the UV coords.
Anyone have any hints on how I would go about getting the UV coords from near the end of the Interior::castRay_r function in interiorCollision.cpp ?
I've overloaded castRay_r to have an alternate version that fills in a TriRayInfo structure in the same way that the resource provided an alternate raycast for a mesh.
I'm getting the correct material ID, but I'm not sure how to go about getting at the UV coordinates for the interior surface although I know which surface it is.
I suspect that if I could get a triangle out of the surface somehow and the UV coords that correspond to it's vertices, I could use the getBarryCentricCoord function that was part of the resource to get the correct UV coords.
For anyone else trying to get this working, GuiControls constructor sets it's bounds to 640x480, so if you create your GUI control at 512x512 to satisfy the power of two requirements for a texture, that will also throw off the hit point calculation. I just set the bounds to 512x512 in the constructor for GuiTextureCanvas, but that's not very satisfying or flexible. Ideally, GuiTextureCanvas sizes its texture to match the size of the GUI you hook up to it, but for now I'm happy that i's working as well as it is.
#88
03/10/2007 (9:57 pm)
Interiors don't have UV coordinates as such. But SceneObject has a method that grabs the diffuse from interiors and uses it to cue object brightness - I'd suggesting looking there.
#89
The following seems to do the trick:
This would be what is needed in the castRay_r function in interiorCollision, although for performance reasons it should be done in a new collision function that is only called when the UV coords are needed. On the other hand, the calc time is pretty constant.. there is no search for anything being done other than what is already done to calc the collision without the UV info.
This may only work on surfaces that have a 1:1 mapping with the texture that is being used, but for placing a GUI on a surface in an interior, you're almost certainly going to map 1:1 onto that surface anyway, so it's good enough for 90% of the cases.
EDIT: This works even when the texture is split across faces, so I was worried about something that isn't a problem.
03/11/2007 (10:23 am)
Thanks again Ben! The light maps turned out to be a dead end, but there was enough stuff in the SceneObject code to figure out something that works pretty much perfectly.The following seems to do the trick:
if(inside)
{
info->face = surfaceIndex;
info->material = rSurface.textureIndex;
MatInstance* matInst = mMaterialList->getMaterialInst( rSurface.textureIndex );
if ( matInst && matInst->getMaterial() )
{
info->material = matInst->getMaterial()->getId();
}
else
{
info->material = 0;
}
const TexGenPlanes &texgen = mTexGenEQs[rSurface.texGenIndex];
info->uv.x = mDot(texgen.planeX, info->point) + texgen.planeX.d;
info->uv.y = mDot(texgen.planeY, info->point) + texgen.planeY.d;
info->uv.x = mFmod(info->uv.x , 1.0);
if (info->uv.x < 0.0) info->uv.x = 1.0 + info->uv.x;
info->uv.y = mFmod(info->uv.y , 1.0);
if (info->uv.y < 0.0) info->uv.y = 1.0 + info->uv.y;
break;
}This would be what is needed in the castRay_r function in interiorCollision, although for performance reasons it should be done in a new collision function that is only called when the UV coords are needed. On the other hand, the calc time is pretty constant.. there is no search for anything being done other than what is already done to calc the collision without the UV info.
This may only work on surfaces that have a 1:1 mapping with the texture that is being used, but for placing a GUI on a surface in an interior, you're almost certainly going to map 1:1 onto that surface anyway, so it's good enough for 90% of the cases.
EDIT: This works even when the texture is split across faces, so I was worried about something that isn't a problem.
#90
03/11/2007 (8:58 pm)
@Pete - For what it's worth when i first got this working i did it with interiors. I believe i did the same thing with mTexGenEQs and it worked just fine.
#91
To answer a concern you had in the early posts, I had a warped DTS mesh with a GUI mapped to it right next to a DIF that had the GUI mapped accross split faces. They both worked and stayed in sync even when they were in the same view, so Dave did a good job of integrating it to play nice with the materials system.
I still have an issue with the players eyepoint being off until a projectile is fired, but all in all it was a good weekend considering it was short an hour.
For anyone interested, here's the rest of the code needed to modify the resource:
That's the modified version of the castRay ConsoleFunction that was already in ther resource.
That's a new overloaded castRay function that takes an InteriorInstance instead of a ShapeBase
The rest of the changes involved propagating a new version of castRay that fills in a TriRayInfo through the InteriorInstance class down through the Interior class and into the castRay_r function.
The new castRay_r is exactly the same as the original, but with the changes above when it returns the info on the colliding surface.
Hope this is helpful to someone else, and thanks to Tom and all the other folks for doing most of the work.
03/11/2007 (9:56 pm)
@Tom - What kept me going was knowing that you'd said near the top you got it working for interiors in about ten lines of code. Whenever I'd get up around 30 lines of code, I'd know I was on the wrong track and go looking for another approach.To answer a concern you had in the early posts, I had a warped DTS mesh with a GUI mapped to it right next to a DIF that had the GUI mapped accross split faces. They both worked and stayed in sync even when they were in the same view, so Dave did a good job of integrating it to play nice with the materials system.
I still have an issue with the players eyepoint being off until a projectile is fired, but all in all it was a good weekend considering it was short an hour.
For anyone interested, here's the rest of the code needed to modify the resource:
ConsoleStaticMethod( GuiTextureCanvas, castRay, const char*, 4, 4, "(ShapeBase object, Point3F start, Point3F end)")
{
ShapeBase* shapeBase;
InteriorInstance* interior;
if ( !Sim::findObject(dAtoi(argv[1]), shapeBase) )
{
if ( !Sim::findObject(dAtoi(argv[1]), interior) )
return "NO";
}
Point3F start, end;
dSscanf(argv[2], "%f %f %f", &start.x, &start.y, &start.z);
dSscanf(argv[3], "%f %f %f", &end.x, &end.y, &end.z);
TriRayInfo info;
if ( shapeBase )
{
if (!GuiTextureCanvas::castRay( shapeBase, start, end, &info ) )
return "NCS";
}
else
if ( interior)
{
if(!GuiTextureCanvas::castRay( interior, start, end, &info ) )
return "NCI";
}
char* result = Con::getReturnBuffer(64);
dSprintf( result, 64, "%d %g %g", info.material, info.uv.x, info.uv.y );
return result;
}That's the modified version of the castRay ConsoleFunction that was already in ther resource.
bool GuiTextureCanvas::castRay( InteriorInstance* object, const Point3F& start, const Point3F& end, TriRayInfo* rayInfo )
{
TriRayInfo info;
Point3F tstart, tend;
MatrixF mat = object->getWorldTransform();
mat.mulP(start,&tstart);
mat.mulP(end,&tend);
rayInfo->distance = F32_MAX;
if (object->castRay(tstart,tend,&info) && info.distance < rayInfo->distance )
{
*rayInfo = info;
// Ok... we got a hit... cast back into world
// space and return it.
MatrixF invmat( mat );
invmat.inverse();
invmat.mulP(rayInfo->point);
}
return rayInfo->distance < F32_MAX;
}That's a new overloaded castRay function that takes an InteriorInstance instead of a ShapeBase
The rest of the changes involved propagating a new version of castRay that fills in a TriRayInfo through the InteriorInstance class down through the Interior class and into the castRay_r function.
The new castRay_r is exactly the same as the original, but with the changes above when it returns the info on the colliding surface.
Hope this is helpful to someone else, and thanks to Tom and all the other folks for doing most of the work.
#92
03/20/2007 (9:07 am)
Heh, just ran into another brick wall. This fails in multiplayer mode using a dedicated server since the server objects have no materials. ouch...
#93
03/20/2007 (12:40 pm)
@Danni - When i implemented it originally i wasn't too knowledgeable about the client/server boundary, but really it should be simple to solve. The trick is to make sure your doing all your raycasts on the client container and not on the server.
#94
I am actually generating a unique hash that is passed from the server to the client when they click somewhere on the GUI, the client sends the key back when the GUI sends a command to the server. The server can then make sure the correct sequence of events occurred and that the unique hash matches each time. In short i am using the code for a computer console in a base within my game that allows the enemy to hack, hijack, etc and the friendly team to see the global status, repair, detect hackers, etc.
I think i have a workaround, which would be to just send the canvas name instead of material id. Going to play with it later. :)
03/21/2007 (7:33 am)
The problem i have there, is i am requiring the server to authenticate what the client is doing and they are not just throwing commands in the console to cheat around the GUI system. I am actually generating a unique hash that is passed from the server to the client when they click somewhere on the GUI, the client sends the key back when the GUI sends a command to the server. The server can then make sure the correct sequence of events occurred and that the unique hash matches each time. In short i am using the code for a computer console in a base within my game that allows the enemy to hack, hijack, etc and the friendly team to see the global status, repair, detect hackers, etc.
I think i have a workaround, which would be to just send the canvas name instead of material id. Going to play with it later. :)
#95
I started implementing this resource but the instructions seem dated and incompatible with TGE 1.5.2
Does anyone have any updated instructions on how to render a GUI on a 3D object?
Thanks for a quick reply
08/22/2007 (2:16 am)
Sorry for cross-posting, but this is really urgent.I started implementing this resource but the instructions seem dated and incompatible with TGE 1.5.2
Does anyone have any updated instructions on how to render a GUI on a 3D object?
Thanks for a quick reply
#96
TGE and TGEA work differently there.
08/22/2007 (5:09 am)
This is a TGEA resource, it will never work with TGE as the rendering core has been replaced.TGE and TGEA work differently there.
#97
08/22/2007 (5:50 am)
So there is absolutely no way to integrate GUI textures with TGE?
#98
Otherwise: no, at least unless I missed the addition of Render2Texture support which is needed for this.
08/22/2007 (8:55 am)
With MK you could try to port that.Otherwise: no, at least unless I missed the addition of Render2Texture support which is needed for this.
#99
And what about the DynamicTexture class? Can't it be used to render GUIs?
08/23/2007 (12:19 am)
What is MK?And what about the DynamicTexture class? Can't it be used to render GUIs?
#100
But you can try if DynamicTexture class works.
Everything is worth a try and depending on your need, nearly anything could work :)
08/23/2007 (1:03 am)
MK is the Modernization Kit, created by Alex, which adds GLSL shader support and a few other things.But you can try if DynamicTexture class works.
Everything is worth a try and depending on your need, nearly anything could work :)
Associate Ben Garney