Mask and Bit Question/Problem
by Chris Fitzgerald · in Torque Game Engine · 09/17/2007 (8:30 pm) · 17 replies
I declare a variable in Shapebase.h:
S32 mSomething;
I set the number of bits up in PublicConstants:
SomethingLevelBits = 4;
I add a Mask
SomethingMask = Parent::NextFreeMask << 8,
In Shapebase.c I initialize the value:
mSomething = 1;
In Packdata:
if(!stream->writeFlag(mask & (NameMask | DamageMask | SoundMask |
ThreadMask | ImageMask | CloakMask | MountedMask |
SkinMask | SomethingMask)))
return retMask;
Then do:
if (stream->writeFlag(mask & SomethingMask)) {
stream->writeInt(mSomething,SomethingLevelBits);
}
And in unpackdata:
if (stream->readFlag()) {
stream->readInt(SomethingLevelBits);
}
The result is that the game crashes halfway through loading the objects. I've used similiar techniques for strings (char arrays, actually) just fine.
I've tried changing the bits and I still get the crash. The variable is just a number between -10 and 10. I should have enough masks -- I've only added three total and I've gotten rid of the Invincible and Shield Masks altogether, so I'm not even close to the limit.
Obviously, there's a packet problem here but, well, I just started working with this last night so I'm not an expert yet :) My hunch is that my SomethingLevelBits is set incorrectly but who knows. Any thoughts?
S32 mSomething;
I set the number of bits up in PublicConstants:
SomethingLevelBits = 4;
I add a Mask
SomethingMask = Parent::NextFreeMask << 8,
In Shapebase.c I initialize the value:
mSomething = 1;
In Packdata:
if(!stream->writeFlag(mask & (NameMask | DamageMask | SoundMask |
ThreadMask | ImageMask | CloakMask | MountedMask |
SkinMask | SomethingMask)))
return retMask;
Then do:
if (stream->writeFlag(mask & SomethingMask)) {
stream->writeInt(mSomething,SomethingLevelBits);
}
And in unpackdata:
if (stream->readFlag()) {
stream->readInt(SomethingLevelBits);
}
The result is that the game crashes halfway through loading the objects. I've used similiar techniques for strings (char arrays, actually) just fine.
I've tried changing the bits and I still get the crash. The variable is just a number between -10 and 10. I should have enough masks -- I've only added three total and I've gotten rid of the Invincible and Shield Masks altogether, so I'm not even close to the limit.
Obviously, there's a packet problem here but, well, I just started working with this last night so I'm not an expert yet :) My hunch is that my SomethingLevelBits is set incorrectly but who knows. Any thoughts?
About the author
#2
Just tested it and everything works. Thanks Stephen
09/17/2007 (9:12 pm)
Ah-ha. I didn't know the order of the read/write calls was critical. Okay, it's starting to make sense now.Just tested it and everything works. Thanks Stephen
#3
In player.h:
GroupMask = Parent::NextFreeMask << 6 ,
In player.c:
____Pack_____
if (stream->writeFlag(mask & GroupMask)) {
Con::printf("WRITE STREAM");
stream->writeString(mGroup);
}
____Unpack_____
if (stream->readFlag()) {
stream->readString(mGroup);
Con::printf("READ STREAM");
}
___Call____
if(isServerObject()){
Con::printf("SETTING MASK");
setMaskBits(GroupMask);
}
This doesn't work for some unknown reasons (I only get "SETTING MASK"). Note that the read and write packs are at the very start of their respected functions, so there isn't a problem with order. In fact, this is how my pack/unpack functions go:
1. GroupMask (does not work)
2. Mask I added (works)
3. Mask I added (works)
4. Normal Player Masks (work)
5. " ", etc
So, it's not even writing the string. Any thoughts?
10/02/2007 (4:32 pm)
It seems like I can get Masks and packing/unpacking data to work most of the time but then there's just some other times when it does not work at all. Here is today's example:In player.h:
GroupMask = Parent::NextFreeMask << 6 ,
In player.c:
____Pack_____
if (stream->writeFlag(mask & GroupMask)) {
Con::printf("WRITE STREAM");
stream->writeString(mGroup);
}
____Unpack_____
if (stream->readFlag()) {
stream->readString(mGroup);
Con::printf("READ STREAM");
}
___Call____
if(isServerObject()){
Con::printf("SETTING MASK");
setMaskBits(GroupMask);
}
This doesn't work for some unknown reasons (I only get "SETTING MASK"). Note that the read and write packs are at the very start of their respected functions, so there isn't a problem with order. In fact, this is how my pack/unpack functions go:
1. GroupMask (does not work)
2. Mask I added (works)
3. Mask I added (works)
4. Normal Player Masks (work)
5. " ", etc
So, it's not even writing the string. Any thoughts?
#4
* print the value of GroupMask to verify it's something < 32.
* put a print statement right *before* writeFlag().
* print the value of mId in setMaskBits() and in Pack().
* you're sure the object is in network scope for the client ?
10/02/2007 (5:00 pm)
Few things to try -* print the value of GroupMask to verify it's something < 32.
* put a print statement right *before* writeFlag().
* print the value of mId in setMaskBits() and in Pack().
* you're sure the object is in network scope for the client ?
#5
If so, I believe those are the functions used by the datablock to transmit the initial data to the client and are only called when the datablocks are transmitted (during the mission loading).
You want to be looking at the packUpdate/unpackUpdate functions. packUpdate is only called when at least one of the bit masks have been set. (So should be called after setting that mask.)
There is also another set of function writePacketData/readPacketData which I believe are used for updating when the object is used as the control object (although I'm not completely sure on this point).
Gabriel
10/02/2007 (6:21 pm)
Are you using the functions: packData/unpackData?If so, I believe those are the functions used by the datablock to transmit the initial data to the client and are only called when the datablocks are transmitted (during the mission loading).
You want to be looking at the packUpdate/unpackUpdate functions. packUpdate is only called when at least one of the bit masks have been set. (So should be called after setting that mask.)
There is also another set of function writePacketData/readPacketData which I believe are used for updating when the object is used as the control object (although I'm not completely sure on this point).
Gabriel
#6
I am using Player::packData and Player::unpackData. From my understanding, these are not used for datablocks -- that would be PlayerData::unpackData and PlayerData::packData.
I have so many weird issues I can't even start. I've only added two masks to Player.c -- a movespeed mask and a jump mask. Thus, I'm not even close to the 32 mask limit, unless that limit applies to all masks in Shapebase, Player, GameConnection, etc all combined (which I don't think it does).
Orion, when I print out my GroupMask, I constantly get a 0. When I print out some (but not all) other masks I get some odd number like -231232332. I'm using: printf("This mask is %d" SpeedMask) so I'm printing the wrong variable type.
I have a function that updates a player's speed. When I stick a printf() in the writeFlag() function, I get a nice printout for that. I also have a function that updates a player's jumpforce using a seperate mask. When I use a printf() function there, nothing prints out. But, I can update both values and they affect the ingame object just fine (ie, I can change the speed and jump for the player character and it doesn't affect the NPCs or other characters).
The group value is just being tied to the player object, so the player should not lose scope.
What a pain :)
10/02/2007 (6:58 pm)
Thanks for the replies you two.I am using Player::packData and Player::unpackData. From my understanding, these are not used for datablocks -- that would be PlayerData::unpackData and PlayerData::packData.
I have so many weird issues I can't even start. I've only added two masks to Player.c -- a movespeed mask and a jump mask. Thus, I'm not even close to the 32 mask limit, unless that limit applies to all masks in Shapebase, Player, GameConnection, etc all combined (which I don't think it does).
Orion, when I print out my GroupMask, I constantly get a 0. When I print out some (but not all) other masks I get some odd number like -231232332. I'm using: printf("This mask is %d" SpeedMask) so I'm printing the wrong variable type.
I have a function that updates a player's speed. When I stick a printf() in the writeFlag() function, I get a nice printout for that. I also have a function that updates a player's jumpforce using a seperate mask. When I use a printf() function there, nothing prints out. But, I can update both values and they affect the ingame object just fine (ie, I can change the speed and jump for the player character and it doesn't affect the NPCs or other characters).
The group value is just being tied to the player object, so the player should not lose scope.
What a pain :)
#7
so in the case of Player, yeah, you've got to add in the masks from Shapebase & GameBase etc.
so GroupMask == 0 is kinda suspicious.
anything << 6 shouldn't be 0 unless you've run out of masks.
try using %0.8X instead of %d with the other masks and you should see nice hex values like 00040000 and such. (eg, printf("mask 0x%0.8X", FooMask))
why does nothing print out in the function to update the player's jumpforce ? that seems odd.
10/03/2007 (12:06 am)
Quote:i've only added two masks to Player.c -- a movespeed mask and a jump mask. Thus, I'm not even close to the 32 mask limit, unless that limit applies to all masks in Shapebase, Player, GameConnection, etc all combined (which I don't think it does).actually you've got 32 masks per class, including all its parent classes.
so in the case of Player, yeah, you've got to add in the masks from Shapebase & GameBase etc.
so GroupMask == 0 is kinda suspicious.
anything << 6 shouldn't be 0 unless you've run out of masks.
try using %0.8X instead of %d with the other masks and you should see nice hex values like 00040000 and such. (eg, printf("mask 0x%0.8X", FooMask))
why does nothing print out in the function to update the player's jumpforce ? that seems odd.
#8
Gabriel
10/03/2007 (4:15 am)
The last used mask in stock player is ImpactMask which calls in at bit number 27. You should have 4-5 bits still available to use in your derivative assuming that 32 is the limit.Gabriel
#9
Test for your self by adding to the player construtor:
Gabriel
10/03/2007 (4:43 am)
Sorry that should read bit 28 (of a range of 1..32). You have only 4 bits available.Test for your self by adding to the player construtor:
U32 maskbit=ImpactMask;
U32 bit=0;
for (U8 i=0; i<32; i++)
{
if ((U32)mPow(2, (F32)i)==maskbit)
bit=i+1;
}Place a break point after the above code and have a look at the value of bit.Gabriel
#10
Quoted for Truth--pack/unPackData have nothing to do with setMaskBits. In fact, they don't even exist in the Player:: namespace--did you add them manually?
Gabriel is correct on all counts by the way in how he describes each pair and what they are used for.
10/03/2007 (7:12 am)
Quote:
You want to be looking at the packUpdate/unpackUpdate functions. packUpdate is only called when at least one of the bit masks have been set. (So should be called after setting that mask.)
Quoted for Truth--pack/unPackData have nothing to do with setMaskBits. In fact, they don't even exist in the Player:: namespace--did you add them manually?
Gabriel is correct on all counts by the way in how he describes each pair and what they are used for.
#11
I thought I had more masks, which explains a few things. I'm going to reorganize my masks as I know I've added more than five total. That should help eliminate some of the problems.
As usual, thanks for the help :)
10/03/2007 (10:40 am)
My mistake. I am using packUpdate and unPackupdate. Staring at a screen for hours does odd things.I thought I had more masks, which explains a few things. I'm going to reorganize my masks as I know I've added more than five total. That should help eliminate some of the problems.
As usual, thanks for the help :)
#12
Many times it's simply not useful to create an entire mask logical group for a single element of data (such as this group thing you are talking about)--either add the new data element to an existing mask group, or use an alternate mechanism to change that type of data.
On that note--it's important to make sure that you are using a rational way of delivering data--for example, the ghosting system is designed for data that is continuously changing, or at least has the potential to change very rapidly in a short period of time. We don't really know what your mSomething (later referred to directly as mGroup) stores--is it something that will only change once, or sporadically, during game play? If it's the equivalent of "being in a group of players" ala most MMO's, then you certainly don't need to use the ghost system for that type of data...explore NetEvents (either a custom one if the data is complex, or simply using commandToServer/commandToClient via script if it's basic) as a better data delivery mechanism.
10/03/2007 (11:27 am)
From a theory perspective, keep in mind that the intent of masks is not a one to one relationship with single elements of data...masks are a compromise for performance optimization that allow you to logically group data sets that are expected to have a high correlation of probability of change (if your x coordinate changes, there is a high probability that your y and z, and even your rotations have changed--so if any one changes, send 'em all).Many times it's simply not useful to create an entire mask logical group for a single element of data (such as this group thing you are talking about)--either add the new data element to an existing mask group, or use an alternate mechanism to change that type of data.
On that note--it's important to make sure that you are using a rational way of delivering data--for example, the ghosting system is designed for data that is continuously changing, or at least has the potential to change very rapidly in a short period of time. We don't really know what your mSomething (later referred to directly as mGroup) stores--is it something that will only change once, or sporadically, during game play? If it's the equivalent of "being in a group of players" ala most MMO's, then you certainly don't need to use the ghost system for that type of data...explore NetEvents (either a custom one if the data is complex, or simply using commandToServer/commandToClient via script if it's basic) as a better data delivery mechanism.
#13
Then there are other values, including run speed and jumping, that I have taken out of the Player datablock. As these can be changed quit frequently and are crucial to proper ghosting, they are in a single Mask. I also added another stat (no, not mana ... yay) for each player. This is in a seperate mask but I may move it to the DamageMask.
Other than that, I've been rather stingy when it comes to Masks. I only want Masks to include data that each client needs to display/render each other client (Name, speed, Skins, etc) that may change over time.
The mSomething in the first post is not related to my current problem, which is indeed related to grouping/squads but moreso for display than anything else. But, I was under the assumption that I had more masks to use, so I'll have to be more judicious with them in the future.
10/03/2007 (12:23 pm)
I have been using Masks mainly to handle updating each ghost with display values that may change during gameplay. These include various titles, clans, etc. Currently, these are piggybacking in Shapebase's NameMask. The frequency of change is quite low, of course, but each ghost needs to be updated to handle the display to clients properly.Then there are other values, including run speed and jumping, that I have taken out of the Player datablock. As these can be changed quit frequently and are crucial to proper ghosting, they are in a single Mask. I also added another stat (no, not mana ... yay) for each player. This is in a seperate mask but I may move it to the DamageMask.
Other than that, I've been rather stingy when it comes to Masks. I only want Masks to include data that each client needs to display/render each other client (Name, speed, Skins, etc) that may change over time.
The mSomething in the first post is not related to my current problem, which is indeed related to grouping/squads but moreso for display than anything else. But, I was under the assumption that I had more masks to use, so I'll have to be more judicious with them in the future.
#14
There is nothing "wrong" with doing data streaming like that of course--I'm not really saying that, I'm just indicating that there are multiple ways to transmit data, and it's good to know the intended use cases for the various means so you can most effectively use them :) Moving data to already existing masks is of course a good strategy, but I will point out why it can be a problem:
During a network cycle, the ghosting portion of a packet is effectively around 200-300 bytes total of data, for all objects requiring an update. Since strings in general require multiple bytes (as opposed to the bits that a properly optimized data set uses within the update) to deliver, grouping multiple strings to all be sent at once if any of them change can flood your update packet, forcing updates to get pushed to the next network cycle once the update packet is filled up.
Again, given the relatively very low frequency of these types of data being changed it's not that big of a deal, but from a purist's perspective it's not really the most efficient use of the ghosting system to be sending string data, which implies a refactor of the delivery mechanism for that type of data to a more appropriate style--in this case, when a string changes, send it as a NetEvent (guaranteed or guaranteed ordered), one event per string, to all clients.
10/03/2007 (12:32 pm)
Yes, quite honestly, I'd be looking at moving to NetEvents for just about all data described in your first paragraph--they simply don't need the ghosting system. Arguably, the shapeName in ShapeBase (NameMask) doesn't even need it--and is probably not an efficient use of bits for ghosting updates.There is nothing "wrong" with doing data streaming like that of course--I'm not really saying that, I'm just indicating that there are multiple ways to transmit data, and it's good to know the intended use cases for the various means so you can most effectively use them :) Moving data to already existing masks is of course a good strategy, but I will point out why it can be a problem:
During a network cycle, the ghosting portion of a packet is effectively around 200-300 bytes total of data, for all objects requiring an update. Since strings in general require multiple bytes (as opposed to the bits that a properly optimized data set uses within the update) to deliver, grouping multiple strings to all be sent at once if any of them change can flood your update packet, forcing updates to get pushed to the next network cycle once the update packet is filled up.
Again, given the relatively very low frequency of these types of data being changed it's not that big of a deal, but from a purist's perspective it's not really the most efficient use of the ghosting system to be sending string data, which implies a refactor of the delivery mechanism for that type of data to a more appropriate style--in this case, when a string changes, send it as a NetEvent (guaranteed or guaranteed ordered), one event per string, to all clients.
#15
10/03/2007 (5:17 pm)
Well, if passing strings using Masks is inefficient, then I'll need to change it when possible. I was looking to use NetEvents for weather which was way down the line -- I'll take a look at them sooner. Again, thanks for the responses.
#16
sounds like you've actually got a fair grasp of networking.
for displaying the stats of other members in your group,
yes, ghosting is a problem because ghosted data is attached to, well, the ghost, which as you point out, may or may not even be scoped to the target client.
you could make the members of a group all scope-always for each other,
or you could do something with commandToClient()s (aka in C++, NetEvents):
when a player first joins a group (including first connecting to the server),
send down all the status for everyone in the group.
then, whenever a group member's status changes,
send just that change to everyone in the group.
10/03/2007 (11:26 pm)
Quote:[i want something that] displays the health and energy of group membershey chris -
sounds like you've actually got a fair grasp of networking.
for displaying the stats of other members in your group,
yes, ghosting is a problem because ghosted data is attached to, well, the ghost, which as you point out, may or may not even be scoped to the target client.
you could make the members of a group all scope-always for each other,
or you could do something with commandToClient()s (aka in C++, NetEvents):
when a player first joins a group (including first connecting to the server),
send down all the status for everyone in the group.
then, whenever a group member's status changes,
send just that change to everyone in the group.
#17
Thanks for the response. You had some good suggestions. I'm still getting used to the engine and making a transition to C++, so I wasn't thinking this through thoroughly. NetEvents/commandToClients should work just fine.
10/04/2007 (4:13 pm)
Orion,Thanks for the response. You had some good suggestions. I'm still getting used to the engine and making a transition to C++, so I wasn't thinking this through thoroughly. NetEvents/commandToClients should work just fine.
Torque 3D Owner Stephen Zepp
--make sure that your packData and unPackData have the matching write and read in exactly the same logical order. It appears that you are writing two bits before you write the int, and then only read in one bit.
--use write/readSignedInt instead of write/readInt.