Introduction

It seems this will become a reoccurring pattern, late last night Findow came on IRC and wanting to know if anybody knew of any working masked button resources for Torque 3D and it happened again. I decided to go in and figure out a working solution myself since there were currently no resources that implemented masked buttons, or what I like to call them shaped mask buttons since they're buttons that are interactively defined by a shaped mask image. Meaning depending on what the shape of the button mask image is defines where the user can interact with the button.

Because of how shaped mask images are used, and luckily how stock Torque's user input works, we now can have overlapping GUI bitmap button controls on top of each other with whatever shapes the developer or artist wants are possible without unwanted effects as seen in this screenshot:

Implementation

First download the zipped archive file and extract it some where:
shapedMaskButton.zip -- Site A
shapedMaskButton.zip -- Site B

Now copy the art/ directory into your project's game/ directory as this will place the GUIButtonStar*.png files where they're suppose to be in game/art/gui/shapedbutton/ directory and also gives you a game/art/gui/mainMenuGui_ShapedButtonDemo.gui GUI file that you can use to test the modification. The zip file also contains a T3DshapedMaskButton20110218.patch patch file in case you prefer not to follow the instructions below for manually implementing the changes.

Usage Notes

Before we get to the engine modification instructions I figured I better go ahead and explain in detail on how to use this correctly for those of you who went ahead and used the patch route.

1. GuiBitmapButtonCtrl has the following rules for image names:
• basename.ext is the normal/unhovered button image.
• basename_h.ext is the highlighted/hovered button image.
• basename_d.ext is the active/depressed button image.
• basename_i.ext is the inactive/disabled button image.
• basename_m.ext is the shaped mask button image.
basename meaning GUIButtonStar in GUIButtonStar.png or GUIButtonStar_m.png for example.
And .ext is the .png file extension which is one of many supported image formats by Torque.


2. When setting guiBitmapButtonCtrl's bitmap field, which is right above the bitmapMode setting in the GUI editor, be sure to not include the file extension as seen in the provided GUI file using this file path to the set of test button images: art/gui/shapedbutton/GUIButtonStar

Torque will automatically go looking for all the supported file extensions, so it shouldn't be a problem as long as you don't have files named like GUIButtonStar.png, GUIButtonStar_m.bmp, GUIButtonStar_m.gif, etc... in the same directory.


3. When creating the shaped mask image file, basename_m.ext, use the color white (color hex #FFFFFF) for user interactive areas and then use black for the non-interactive area of the mask image to make easier to tell during image editing what parts of the button should be user clickable. This modification specifically looks for white colored pixels to determine if the mouse pointer is over pixel area that is allowed for user interaction, else all other colors are ignored.

One last thing, you can rename your existing game/art/gui/mainMenuGui.gui file to like mainMenuGui_original.gui and then rename the provided mainMenuGui_ShapedButtonDemo.gui to mainMenuGui.gui to immediately test out the new shaped mask buttons. If you've just went ahead and applied the patch then you can test it immediately that is, else proceed on to manually modifying the guiBitmapButtonCtrl source files as explained below to finish the implementation. And then you may check it out. :)

Source Code Changes

Open the file engine/source/gui/buttons/guiBitmapButtonCtrl.h for editing. Now SEARCH for these two lines:

/// Texture for inactive state.
GFXTexHandle mTextureInactive;

and INSERT this snippet below them:

/// Texture for shaped hitmask.
GFXTexHandle mTextureMask;

Again SEARCH for these two lines:

virtual void onAction();
virtual void inspectPostApply();

and INSERT this code block below them:

/// This function will return true if the provided coordinates (wrt parent object) are
/// within the bounds of this control
/// @param   parentCoordPoint   Coordinates to test
virtual bool pointInControl(const Point2I& parentCoordPoint);

Now done with the header file, so save it and then open the file engine/source/gui/buttons/guiBitmapButtonCtrl.cpp for editing. SEARCH for this line:

static String s_i = "_i";

and INSERT this below it:

static String s_m = "_m";

Within the same function SEARCH for these three lines:

mTextures[ i ].mTextureInactive = GFXTexHandle( baseName + s_i, &GFXDefaultPersistentProfile, avar("%s() - mTextureInactive (line %d)", __FUNCTION__, __LINE__));
if( !mTextures[ i ].mTextureInactive )
  mTextures[ i ].mTextureInactive = mTextures[ i ].mTextureNormal;

and INSERT this line below them:

mTextures[ i ].mTextureMask = GFXTexHandle( baseName + s_m, &GFXDefaultPersistentProfile, avar("%s() - mTextureMask (line %d)", __FUNCTION__, __LINE__));

Again within the same function SEARCH for this line:

mTextures[ i ].mTextureInactive = NULL;

and INSERT this line below it:

mTextures[ i ].mTextureMask = NULL;

Now for the grand finale, INSERT this function at the end of the source file:

//-----------------------------------------------------------------------------

bool GuiBitmapButtonCtrl::pointInControl(const Point2I& parentCoordPoint)
{
	GFXTexHandle		*texture	= NULL;
	GBitmap				*bmp		= NULL;
	U32					index		= ModifierNone;


	/// abort when coordinates aren't even within control rectangle boundaries
	if(!Parent::pointInControl(parentCoordPoint))
		return false;

	if(mUseModifiers)
		index = getCurrentModifier();

	// abort shaped mask hit test if no shape mask image loaded
	texture = &mTextures[ index ].mTextureMask;
	if(!texture || !*texture)
		return true;

	// get shaped mask bitmap, if available, else abort out
	bmp = texture->getBitmap();
	if(!bmp)
		return true; // TODO: shouldn't we report a warning of somekind? --TRON

	/// determine if point in control is hitting shaped mask
	const RectI		&bounds		= getBounds();
	S32				xt			= parentCoordPoint.x - bounds.point.x;
	S32				yt			= parentCoordPoint.y - bounds.point.y;
	ColorI			rColor(0x00, 0x00, 0x00);
	ColorI			rMask( 0xFF, 0xFF, 0xFF);

	// correct the test coordinates depending on drawing mode
	switch (mBitmapMode)
	{
		case BitmapStretched:
		{
			F32		rw, rh;
			S32		px			= getExtent().x;
			S32		py			= getExtent().y;

			// prevent divide by zero
			if(!px) px = 1;
			if(!py) py = 1;

			// calculate stretched bitmap ratio
			rw = (F32)texture->getWidth()  / px;
			rh = (F32)texture->getHeight() / py;

			// correct the coordinates by ratio
			xt = (S32)(xt * rw);
			yt = (S32)(yt * rh);
			break;
		}

		case BitmapCentered:
		{
			xt -= getExtent().x / 2 - texture->getWidth() / 2;
			yt -= getExtent().y / 2 - texture->getHeight() / 2;
			break;
		}
	}

	// get pixel color from shaped mask with provided coordinates
	bmp->getColor(xt, yt, rColor);
	rColor.alpha = 0xFF;

	// when colors match then the point did hit shaped mask area, else it didn't
	return (rColor == rMask);
}

Save this source file and you're all done with engine modifications. Now compile and build the Torque engine, and as long as you did everything as instructed above then it should build and link successfully.

Results

If you renamed/moved your original mainMenuGUI.gui and used the one provided in its place you should see something like this:

Enjoy!

Credit goes to Charles Doty for his forum posting Masked Bitmap Buttons the torque way :) that pointed me in the right direction to come up with this resource.