Performance with V1.5

#1 Just as a side note, the server runs fine, its only the client that suffers (and yes, i have tried the machines the other way round). I also tried other games on the net, and they are fine.

#2 Are you talking about Torque Game Engine or TNL? Did you acccidentally turn on adaptive networking?

#3 Yes, its TNL im talking about.

Funny you should mention adaptive networking, it runs much better if i call setIsAdaptive on the connections.

Having spent the day on this, i have managed to get the networking back to its original speed, although this involved changes to my surrounding net/game interface rather than the actual TNL settings.

Having said all this, currently I am sending far too much data, around 60 entities each requiring around 350bytes, plus some other minor calls using your RPC mechanisms.

I know this is too much data for a realistic game on the internet (i'm just getting it all working at the mo), with 1.4 it did cope quite well on our LAN, which prompted me to ask these questions.

I have looked back at a number of previous posts, and one was regarding an issue I had today. I attempted to alter the default bandwidth from 2.5Kb per sec to 50k, and subsequently changes to a number of other values. Each change resulted in bad packets being received by the client (sometimes causing disconnections through invalid packets, and othertimes seeing unpackupdates for objects that should not be updating on the network!).

I take it from my results this issue has not been resolved, or is it a case that the numbers must be massaged around each other, and not just a simple case of changing one independantly?

If the case is that you must allow for changes in one value by altering the others, could you document/tell me how they correspond?

Currently then I am getting best preformance with 1.5 TNL by setting it to adaptive (albeit with grossly overweight data issues) and currently not controling how many entities i'm transmitting to each client (clients get all other entities data regardless of how far away they are).

You mentioned in your reply "Did you acccidentally turn on adaptive networking?", does this mean its a bad thing? or where you looking for inconsistancies that i might try to restore?

cheers,
M

#4 I'll let Mark field this one. :)

#5 Hey M,

You almost certainly should _not_ use the adaptive connection for simulation. What values did you set your fixed rate parameters to? I'm not sure why you're getting invalid packet errors, however.

Also, are you using RPCs to update all the objects in the world, or are you using the ghosting system (GhostConnection)?

- Mark

#6 Mark,

Having made some more changes, it now works better than it did in non adaptive, although this took some data reduction and game data fetching speedups as opposed to actual TNL changes.

For transmitting positions and other high rate info, i use ghostConnections, but for important stuff such as applying damage etc i use the RPC calls.

The values i am setting for the fixed rate are:


DefaultFixedBandwidth = 5000
DefaultFixedSendPeriod = 96
MaxFixedBandwidth = 65535
MaxFixedSendPeriod = 2047

The problems arise if i raise the DefaultFixedBandwidth up from 5k (std is 2.5k).

I would expect to get a sustained 50k and higher on a LAN, but if i set DefaultFixedBandwidth to 50k, the client starts receiving bad packets and bails out. I have checked the packets to see if there are any write/read mismatches that might cause this problem, but its fine, and it works fine when set to 5k.

All other parameters are stock, standard udp buffers, everything. 2.5k is not enough for what i am doing, and even a dial up could give more bandwidth than that.

If i transmit diectly with sendto(), then i can transmit much greater amounts of data, and its done safely, so its not a problem with my network and/or data.

The version i am using is the downloadable 1.5 TNL from the site, and not from the CVS Rep.

cheers,
M

#7 The bugs causing the invalid packet probem have now been fixed, tested and checked in to CVS. The fixes should propagate to the public CVS soon.

I was getting a sustained 60K on my local machine by setting the DefaultFixedBandwidth to MaxFixedBandwidth and the DefaultSendPeriod to 20.

#8 @Mark

Could you specify:

1) The exact link where you have your update

2) Exactly which TNL files you have modified - I have made modifications to TNL and at this point I do not have the time to make a detailed comparison of the different versions.

Thanks,

Fabian.

#9 Fabian,

The files were checked in to the SourceForge CVS for TNL -- see sourceforge.net/cvs/?group_id=106342 for details. You can use CVS to determine what files have been changed and get diffs. Also, there's a mailing list - sourceforge.net/mailarchive/forum.php?forum_id=40303 that is pinged every time anything is changed in CVS.

- Mark

#10 @Mark

Ok - will look into it. Thanks for the prompt response on this.

- Fabian

#11 Sounds like a fix!

cheers guys

#12 @Mark

I am back to square one in terms of the window being full. This happens almost immediately, as soon as a connection is established. Our deadline is in less then two days, and I simply cannot get the networking done, because the window is full as soon as I startup the connection.

I am not doing anything to slow down the connection, am using fixed rate parameters :

setFixedRateParameters(50, 50, 5000, 5000);

As a result of the window being full we immediately upon connection, we get a lag effect on our LAN. I am at loss at what to do about this. Packets are sent, the sequences are acked, nothing is missed out and the lag time gap increases substantially.

Have you modified anything in CVS to do with bandwidth / performance related issues in the last month?

We are in a dire situation here so any feedback would be appreciated.

#13 @fabian

do you know how much data your actually sending?

could it be that your sending more data than the network can pull from the send buffers per second?

I had a problem right at the start where i was sending so much data (well, trying to), it could not cope, the client suffered because of this.

Another problem I had was when the clients framerate dropped below the send frequency (heavy GFX and AI load in debug running from within .net), this caused network probs too (massive stutters and positions jumping huge distances).

I ran our game with your setFixedRateParameters values, it was fine. I am sending roughly just under 5k per second (but in adaptive it reaches around 9k) and its fine.

#14 Hey Fabian,

Make sure that your NetConnection subclass is returning true from isDataToTransmit() - i.e. override the function and just always return true.

The only modifications I've made in the last month were to fix the problems M was having. How long has this been a problem, and can you describe the problem in more detail? Why is the window full on startup? Do you have the same problems with the test program?

You can email me at markf (at this website :) ) and we can discuss in greater detail.

- Mark

#15 @Mark

cheers for that update, finally got a chance to get the latest from CVS and it works nicely.

#16 @M

Indeed, it turns out that our frame rate, which dropped below the amount of packets being sent was the main culprit of this windowfull() returning true. So if I set the params to:

setFixedRateParameters(96, 96, 5000, 5000);

the window does not fill up (at least not immediately). The more I increase the send / receive parameters, the less this problem is likely to occur, but then again, my response time increases accordingly as less packets are being sent.

@Mark

The problem I see with an adaptive connection is that the frame rates must match exactly on both machines. Thats why the window returns true almost immediately if i use this as packets are most likely sent all the time, or at least as fast as the game loop runs, which can vary on different machines. Correct me if I am wrong on this one.

Is there not a way to find a compromise between both options - that is, send more packets if the if the sequence numbers get out of whack (on both ends of the connection, of course) and then stabilize it when they are back in sync?

I am looking into making comparisons in NetConnections sequence related values and comparing them to the actual game frame rate to see if I can achieve this. This is all very low level though, and if I have to get my feet wet with this, then I won't have a choice but do achieve it manually.

Was hoping there was a higher level solution to this problem but it looks like I will have to tackle it first hand.

#17 The adaptive code does not work properly. I would avoid using it unless it is for a very low bandwidth connection -- like to a chat server or something like that. I will revisit this part of the codebase at some point to get it working well.

If you want to take a crack at getting the adaptive code rocking, and come up with something cool, let me know. Right now it looks like the adaptive mode doesn't set the packet size at all -- which it should (up to probably MaxDataPacketSize or at least the MTU for UDP which is like 576 bytes or something). All the cwnd and ssthresh stuff should go away as well -- I'm thinking it should have a target bandwidth value, compute the time between packets from that, and then use the packetDropped/packetReceived to modulate the target bandwidth. It would also be cool to be able to set, on a NetInterface, the maximum bandwidth for all the connections and have them adapt properly across an entire server.

Just some thoughts...
-Mark

#18 Yep - this all sounds like good stuff to have. Either way, its looking like I am going to have to modify the existing framework, that is based in fixedRateParameters, and have it send packets more frequently when the connection is out of sync. Will keep you posted on if I come up with something interesting.