1024 Ghost limit/Incr GhostConstants gives out of range err

#1 Did you make sure that the bitsizes are all properly set?

#2 I have only modified the GhostConstants enum Ben. I started looking thru this code today so Im very unfamiliar. I would assume that anything that uses the ghost tables/id or look up tables would use these constants for thier sizes but who knows. Can you enlighten me on this at all?

Thanks

CG

#3 Well... it SHOULD be using GhostConstants everywhere... but it might not be. :-/ I'd see what line was borking on you.

#4 Its croaking at: BitStream::readFlag()(bitstream.h)

if(bitNum > maxReadBitNum)

bitNum is 1761
maxReadBitnum is 1760

And the stack:

torqueDemo_DEBUG.exe!BitStream::readFlag()  Line 158	C++
torqueDemo_DEBUG.exe!HuffmanProcessor::readHuffBuffer(BitStream * pStream=0x00b681f0, char * out_pBuffer=0x0012f199)  Line 692 + 0x8	C++
torqueDemo_DEBUG.exe!BitStream::readString(char * buf=0x0012e950)  Line 537	C++
torqueDemo_DEBUG.exe!Stream::readSTString(bool casesens=false)  Line 65 + 0x14	C++
torqueDemo_DEBUG.exe!WaterBlock::unpackUpdate(NetConnection * c=0x0176e6a0, BitStream * stream=0x00b681f0)  Line 886 + 0xa	C++
torqueDemo_DEBUG.exe!GhostAlwaysObjectEvent::unpack(NetConnection * ps=0x0176e6a0, BitStream * bstream=0x00b681f0)  Line 76 + 0x1b	C++
torqueDemo_DEBUG.exe!NetConnection::eventReadPacket(BitStream * bstream=0x00b681f0)  Line 290 + 0x15	C++
torqueDemo_DEBUG.exe!NetConnection::readPacket(BitStream * bstream=0x00b681f0)  Line 656	C++
torqueDemo_DEBUG.exe!GameConnection::readPacket(BitStream * bstream=0x00b681f0)  Line 793	C++
torqueDemo_DEBUG.exe!NetConnection::handlePacket(BitStream * bstream=0x00b681f0)  Line 521 + 0x11	C++
torqueDemo_DEBUG.exe!ConnectionProtocol::processRawPacket(BitStream * pstream=0x00b681f0)  Line 225 + 0x11	C++
torqueDemo_DEBUG.exe!NetConnection::processRawPacket(BitStream * bstream=0x00b681f0)  Line 494	C++
torqueDemo_DEBUG.exe!NetConnection::sendPacket(BitStream * stream=0x00b681f0)  Line 638 + 0x2b	C++
torqueDemo_DEBUG.exe!NetConnection::checkPacketSend(bool force=false)  Line 622 + 0x11	C++
torqueDemo_DEBUG.exe!NetInterface::processServer()  Line 413	C++
torqueDemo_DEBUG.exe!DemoGame::processTimeEvent(TimeEvent * event=0x0012fc14)  Line 662	C++
torqueDemo_DEBUG.exe!GameInterface::processEvent(Event * event=0x0012fc14)  Line 71 + 0x11	C++
torqueDemo_DEBUG.exe!GameInterface::postEvent(Event & event={...})  Line 153 + 0x11	C++
torqueDemo_DEBUG.exe!TimeManager::process()  Line 1225 + 0x17	C++
torqueDemo_DEBUG.exe!DemoGame::main(int argc=6, const char * * argv=0x01201740)  Line 481	C++
torqueDemo_DEBUG.exe!run(int argc=6, const char * * argv=0x01201740)  Line 1142 + 0x1a	C++
torqueDemo_DEBUG.exe!main(int argc=6, const char * * argv=0x01201740)  Line 1178 + 0xd	C++
torqueDemo_DEBUG.exe!mainCRTStartup()  Line 259 + 0x19	C
KERNEL32.DLL!7c4e87f5()

#5 I tried changing that same enum and I'm not having any problems. Are you sure you are running HEAD or Release_1_2_0 of the engine?

#6 Good point. Im using HEAD download Aug 28. Ill try new version. Thank you

CG

#7 Whew! I downloaded the new head from cvs only to find out that my vs.net doesnt open the solution file anymore. Guess GG has upgraded to 2003. I dug up vc6 and installed and built everything. I made the same change to sim/netconnection.h:766

enum GhostConstants 
   { 
      MaxGhostCount = 4096,
      GhostIdBitSize = 12, 
      GhostLookupTableSize = 4096
   };

And then ran the new FPS starter. I now get a friendly error message:

Disconnect:
You do not have the correct version of the FPS starter kit or the related art needed to play on this server, please contact the server operator for more information (invalid packet)



I put my break at:

if(mErrorBuffer[0])
      connectionError(mErrorBuffer);   << break

This is the stack:

NetConnection::handlePacket(BitStream * 0x00b5f430 gPacketStream) line 516
ConnectionProtocol::processRawPacket(BitStream * 0x00b5f430 gPacketStream) line 225 + 17 bytes
NetConnection::processRawPacket(BitStream * 0x00b5f430 gPacketStream) line 487
NetConnection::sendPacket(BitStream * 0x00b5f430 gPacketStream) line 631 + 34 bytes
NetConnection::checkPacketSend(unsigned char 0) line 615 + 17 bytes
NetInterface::processServer() line 413
DemoGame::processTimeEvent(TimeEvent * 0x0012fe60) line 662
GameInterface::processEvent(Event * 0x0012fe60) line 71 + 17 bytes
GameInterface::postEvent(Event & {...}) line 153 + 17 bytes
TimeManager::process() line 1225 + 23 bytes
DemoGame::main(int 4, const char * * 0x01252f30) line 481
run(int 4, const char * * 0x01252f30) line 1142 + 26 bytes
main(int 4, const char * * 0x01252f30) line 1178 + 13 bytes
TORQUEDEMO_DEBUG! mainCRTStartup + 197 bytes
KERNEL32! 7c4e87f5()

Thx

CG

#8 Sounds like something still isn't getting updated properly :)

The bitstream class lets you specify how many bits to encode an integer in. All code SHOULD be using the constant from the enum, but if some isn't, it might throw the packets off, resulting in that error. You need to check and see if any of the data that's making you bork was read using a hardcoded bit size.

#9 Ok cool that helps, thanks ben.

CG

#10 Well I found 1 place that should be using the Ghost enum that isnt:

sim/netGhost.cc @ 59

replace:

ghostIndex = bstream->readInt(10);

with

ghostIndex = bstream->readInt(NetConnection::GhostIdBitSize);

Now I have the ghost constants change to 12 bits, 4096 count. I still get errors and am thinking that it may have something to do with the SimObject ids. If you look at

console/simBase.h @ 44

You will see there are also constants for simobjects that are setup as 10 bits....

Anyone out there looked at this?

CG

#11 I think its worth the note that I dont have any problems until I have over 1024 obects on my map.

CG

#12 I don't know right offhand, but Torque might be using a variable length encoding for the bits. Dunno. :)

Anyway, yeah, you're going to have to track down all the places that assume 10bit field sizes and update them to the right length. :-/

#13 Well Im still humping this NetObjects/Ghosts limit... I found some other places where the ghosts are not using the constant. All of them were using readInt(10)/writeInt(10) changed to (NetConnection::GhostIdBitSize)

1. game/camera.cc (357)
2. game/gameConnection.cc (747)
3. game/gameConnection.cc (900)
4. game/item.cc (816)
5. sim/netGhost.cc (59)

And some others for total = Bitsize + 1. Were readInt(11)/writeInt(11) changed to (NetConnection::GhostIdBitSize + 1)

6. sim/netConnection(49)
7. sim/netConnection(55)
8. sim/netConnection(61)


It all compiles and runs without error. But its still only showing 1024 objects. I can see the other objects if I switch to MissionEditor mode. What gives??

CG

#14 Does anyone have any more advice on this problem?

I have run into this as well!

#15 I tihk I managed to break it. I made the changes you specified, and the World editor allows up to add more then 1024 with no problem. We are working on a single playter game, so I upped it to 16 bits, giving us 65k of items available.

#16 I think there at least one more thing (besides changing the constant and any readint(10) writeint(10) that are applicable:

change NetGhost.cc around line 451 (in ghostReadPacket):

// PFR: Ghost issue
   S32 idSize;
   idSize = bstream->readInt( 4); // updated from 3 to 4 -- upto 19 bit number
   idSize += 3;

change NetGhost.cc around line 354 (in ghostWritePacket):

// PFR Ghost:
   bstream->writeInt(sendSize - 3, 4); // updated from 3 to 4

That's the only other change I can think of. I've only tried
it on a localclientconnection.

-Pascal

#17 Thanks!

#18 I was able to add more than 1024 items to the map with out it crashing, but it just wasnt rendering more than 1024. Did you get this working?

CG

#19 @CG

Do you have the change above (idSize)?
I was able to get more than 1024 ghost objects visible. The only changes I believe I made was changing the constants in sim/netConnection.h to:

enum GhostConstants 
   { 
      MaxGhostCount = 65536,
      GhostIdBitSize = 16, 
      GhostLookupTableSize = 65536
   };

and then in addition to my comment above, change all the ghost read and writes to use NetConnection::GhostIdBitSize instead of a hardcoded 10 for the bitsize.

-Pascal

#20 Same here. I did EXACTLY the same thing, and then applied Pascal's other small fix.

Everythign seems to work fine. I can place and see more than 1024 items, without a crash.... so far ;)