Removing the 32 bit limit for packUpdate.

Download Code File

So you have finally hit the wall and found that the 32 bits that have been provided for in packUpdate are just not enough. You probably even looked into changing it, but discovered it's not an easy change to make. Well this resource will let you break that 32 bit barrier and put in as many as you want.

However, first a warning.. If you go down this road you will be far off the beaten path of a standard code base. You will not be able to easily merge in others resources, packs, etc to your code base. If you don't feel like you could do such a wide ranging change yourself then you probably shouldn't be doing it.

My major hope is that GG will adopt this (or a similar system) as part of the base so that this artificial limitation can go away. Admittedly it does help enforce the real limitation of limited bandwidth in a networking situation.

Now on to describe the changes.

First I built a specialized BitSet class called PackBitSet. I made it look similar to the STL bitset class.

This class DOES NOT support conversion to/from a U32 bitmask. I highly recommend never allowing that to occur since it will only lead to bugs and unsupportable code down the road for you. It does support a range constructor (U32 bitStart,U32 bitEnd), a bit array constructor (U32 bitlen, U32 bits[]), and a composite constructor (U32 masklen, PackBitSet masks[], U32 bitlen, U32 bits[]). These support construction of constant masks. I also included a helper macro to auto size static arrays.

The next thing I did was renaming/revaluing ALL of the constants that head towards the PackBitSet. I did this to help myself merge in outside resources. If something is referencing InitialUpdateMask I know it's old code and I need to convert it over to the new way. I highly recommend doing that since the type of the constants aren't changing, but the values are changing greatly (1 << bitID vs bitID). I appended _BITID to all the simple bit id's.

For those cases were a true mask was required (ThreadMask, SoundMask, ImageMask) I declare them as extern const PackBitSet's and appended _BITMASK to them. The actual static const definition had to be put in the respective .cc file though.

There are alot of them and I'm not going to list them all here, examine the patch file for the details.

Next the class members we will be changing the type/prototypes of:

netObject:
U32 mDirtyMaskBits turns into
PackBitSet mDirtyMaskBits_PBS

setMaskBits(U32) turns into 2 functions
void setMaskBits_BITMASK(const PackBitSet &orMask);
void setMaskBit_BITID(const U32 orMask);

clearMaskBits(U32) turns into 2 functions
void clearMaskBits_BITMASK(const PackBitSet &orMask);
void clearMaskBit_BITID(const U32 orMask);

For the set/clear we typically only operate on 1 bit. For those cases that operate on 2, I just call it twice.
If they operate on 3 or more I created a static mask.

virtual F32 getUpdatePriority(CameraScopeQuery *focusObject, U32 updateMask, S32 updateSkips) turns into
virtual F32 getUpdatePriority(CameraScopeQuery *focusObject, const PackBitSet &updateMask, S32 updateSkips);

virtual U32 packUpdate(NetConnection *conn, U32 mask, BitStream *stream) turns into
virtual void packUpdate(NetConnection *conn, const PackBitSet &mask, PackBitSet &retMask, BitStream *stream);

You will noticed that instead of returning a PackBitSet it carries along it's retMask as a reference.
It also carries the incoming mask as a const reference. In theory that shouldn't be a problem, however a few

places do cear mask bits, but it is an network optimization local to that function so I just made a local copy of

the const mask in those situations.

netConnection:
U32 mask turns into
PackBitSet mask_PBS

U32 updateMask turns into
PackBitSet updateMask_PBS;

lightUpdateGrouper.h
U32 LightUpdateGrouper::BitIterator::getMask() turns into
U32 LightUpdateGrouper::BitIterator::getMask_BITID()

U32 LightUpdateGrouper::getKeyMask(const U32 key) const turns into
U32 LightUpdateGrouper::getKeyMask_BITID(const U32 key) const

LightUpdateGrouper was returning a bitmask, now it will be returning a bit id. Again a place where the name change will draw attention to any code that is expecting the old behavior.

The next step is to search all the .h files for subclasses implementing these methods and fixing the declarations.

Then you have to go through all the .cc files and correct the code to work under the new semantics. mask & InitialMask turns into mask.test(InitialMask_BITID), etc.

When your all done you should have a working torque that no longer has the 32 bit limitation. I set it up for a 64 bit limit now, but it's just a simple constant in PackBitSet.h so you can expand it out if you decide you really need all those extra bits.

As a side effect of the switch from mask to bitid the pathcamera.h mask bug gets fixed.

I did find and fix a subtle potential bug when calling clearMaskBits. If you were trying to clear multiple bits and not all of them had been set and this would empty out the updateMask then walk->connection->ghostPushToZero(walk) would not get called. i.e. clearMaskBits(3) when updateMask=1 would not do the push.

To apply the patch file go to your code directory and execute:
patch -p1 -u
It doesn't update any makefiles, etc so you will have to add engine/core/PackBitSet.cc manually for your build environment.
This should work for any platform, but I have only tested it on Windows.