Sending variable length arrays?

#1 You might check out the tutorial on picking, actually.
it shows how to insert conditional data into the stream.

#2 It's pretty simple. At some point you're going to get your hands on a BitStream (either in your NetEvent's write method or in a packUpdate), and then you write out... well... whatever data you want. :) The naive approach would be an array length and then all the data. If you're more clever it's going to be some encoding specific to your datastructure and some shared state between server and clent. Maybe an update structure when a change happens.

Bear in mind packets max out at 1400 bytes or so, so you will have to chop data up if it gets bigger than that.

Do you understand how NetEvents work?

#3 No, I haven't looked much into NetEvents, although I came across them trying to dissect the commandToServer functions. I came up with a way to represent the data that will be less than 1400 bytes on an average case, although it could get up to 10000 or more. So I could do this using just commandToServer/client (in c++ of course). I suppose using NetEvents would be a good way to go if I understand them. I do have a couple more questions about this now, though. If I have to chop up the data, how do I stitch it back together? Does commandToServer support char values 0-255 in the string, or is it limited to alphanumeric chars?

Well, before I go on, I thought about datastructures and sending updates only. I'll tell you what I'm planning to do. The 258x258 map will really only need representation on 256x256 of those. There will only be 1's and 0's in this particular map, so each could theoretically be represented by a bit, but my idea may give better compression in general. The 1's will be fairly sparse, and grouped together in blobs. So I will scan through the map, find the start of a row, and count how many "pixels" are in the run. I'll store the starting pixel coords, then how many are in the run. So it's really just RLE, but specific to having just 1's and 0's. Each "run" needs 3 bytes to store. This map needs to be updated at least once every 2 seconds, preferably once/second, so it must be lightweight.

There's another map I need to store though, and that will need to have 8-16 different "colors", which are also all in nice big regions. It's also 256x256. So RLE is a good choice, but it will still be more than 1400 bytes per update with my current representation. This particular map is only needed to transfer once per session from server to client. In fact, it could just always be stored with the client, but I want to avoid cheaters where I can. Perhaps something like an MD5 sum would do to check if an update is necessary.

I also thought about quadtrees, but I think the overhead is greater than necessary for this application.

With that explanation, would you think it's better to create a structure and use packUpdate or NetEvents and such? Or should I just send it in an array (i.e., y-coord, x-coord, count)? Which will be more network efficient in the long run?

If only I had a few months to wait... I'm taking a networking class at the university right now. Some of that may help out, but I'm just not going to wait. The time to write this game is now.

#4 I read through the documentation on NetEvents and BitStreams (thanks for the keywords, Ben) and came up with a better format, I think.

Each row in the maps above will be atomic, and marked by a BitStream flag. The server will store a current version of the map as well as the last thing sent over the wire. Each time it needs to send out again, it will do a diff on each row between the two versions of the map. It will only send the necessary rows.

This does two things.
1. Sends only needed updates
2. Eliminates y-coordinate, since each row will be prefaced by a flag

Of course, I will also need to send the number of runs along each row, which will nearly always be under 32 (that's a high number) and it's theoretically impossible to go over 128. So I can write out a ranged U32 to whatever size I feel necessary. So it saves a few bits in general.

Currently I have the object that will do all of this as a ConsoleObject. It will store the maps needed as well as other information. The question is, should I keep this as a ConsoleObject and just make NetEvent classes to handle sending the things it needs, or should I create a NetObject or NetObjects? Intuition says I should keep it as a ConsoleObject and use NetEvents, but I want to see the benefits of both sides.

Thanks for listening to the monologue :)

#5 Be aware that if you go over the 1400 byte limit, Torque will assert and die.

NetEvent is a SimObject subclass is a ConsoleObject subclass. For this sort of thing, esp. if there's only one, I'd say, stick with the events.

#6 I'm ready to start implementing this now, and came up with some better ideas, and a couple questions.
There is one map that will need to transfer at the beginning of each mission, which may not have great compression. This is the "ownership" map, and specifies which players occupy which areas of the map. This needs to be sent from the server to all clients (up to 16, I think) at the start of the game.
I discovered a different method to handle the map that needs constant updates. Instead of sending the map, I will have each client compute it at the specified updates, then the server will compute it at the end of each round and transfer it to clients (to discourage cheating). What needs to be transferred often now are construction orders. That consists of about 4 bytes of data (if packed in a BitStream). Each time a player builds a wall, these orders need to be sent to the server, then from the server to all clients.
Now the question is again whether I should use NetEvents or NetObjects. Each client needs a place to store maps (there are 3 so far) and a way to send/receive maps. The server also needs this. Looking at NetObject more closely, it seems like a better way.
The next question is, should I put all of this in one NetObject? If so, how can I tell which type of message is being sent and what to do with it? It seems like I should create a separate NetObject for each map, then point to the maps (inside NetObjects) using a SimObject derviative to do calculations on them. Is this wise?
In the above idea I mean to have something like:

WallObject : NetObject {
};
OwnerObject : NetObject {
};

ManagerObject : ConsoleObject {
     WallObject *wall;
     OwnerObject *owner;
};

When setup this way, does Torque magically send the right data to the right NetObject? Of course I need to have packUpdate / unpackUpdate built properly. But do I have to manually send / receive data, or does Torque do that?

I really should just jump into this, but I'm hesitant because it seems rather complex, and I want to approach it the right way the first time. Plus, I think this information might be helpful to others along the way.

#7 Quick answer to your question:

Quote:When setup this way, does Torque magically send the right data to the right NetObject? Of course I need to have packUpdate / unpackUpdate built properly. But do I have to manually send / receive data, or does Torque do that?

TGE does this "auto-magically", but only if the object is set as scopable, and the scope query code has a way to "detect" each object properly.

Scoping is normally done for each object based on the camera postion in relation to the object (see ::onCameraScopeQuery() ), but of course you can implement any scoping rules you want for your objects.

#8 Ok,
Thanks for the quick response. So I just need to set Ghostable and ScopeAlways flags for it to be "visible."
That helps. Now is the rest of my thinking wise?

#9 So I tinkered a little by creating a test setup. I have a SimObject and a Netobject. The netobject is inside the Simobject. I have a consoleMethod on the SimObject that calls setMessage in the NetObject. The netobject is the simple setup inside nettest.cc. When I call setMessage on the SimObject, nothing happens (it's supposed to print to the console). When I call setmessage on the NetObject, it works properly. So something's not working right.

I need the communcation between the manager object and the data stored in the NetObjects to be fast, as I'm doing some moderately intensive algorithms on them (they took 12 seconds to do in script, and almost nothing in c++). So I can't simply communicate via console.
One idea just came to mind, but I'm not entirely sure how to do it. I noticed the resolveGhost functions, so I could create the NetObjects and the Manager object simply in the mission file. Then when the manager needs info from them, it resolves the ghost and gets the data.
It seems like this method will need to transfer stuff across the network anytime I need to access data, even if it doesn't need to be completely up to date (like the contained area map) like things that will be computed on the client side.

Let's try this idea. The Manager : SimObject class simply will contain methods that send/receive netEvents. One such netEvent will send/recv a build command. Another NetEvent will synch the contained Area map. Another will synch the ownership map. So there would be 6 methods, 3 to send, 3 to receive. There would also be 3 different NetEvent derivatives. The Manager object will then store the actual map data itself, and updates will be handled by manually sending NetEvents.
Now the trouble I'm having is how to actually create the Manager object on the client and on the server and how to get them talking. I noticed the Pathmanager class, and it seems to do more or less what I want, but after studying it for about half an hour I'm still somewhat lost on how it tells the client and server apart. I noticed it creates two global variables gClient and gServer in short, and sets isServer to true on the server and false on the client. Hmm... talking about it gave me an idea. The unpack/process is only used to update the client, and the pack is only used to send from the server.
Now if a buildEvent (my class) is used both by the server and the client (client sends command event to server, server responds giving the OK), do I need to split this into two events, buildEvent and buildResponseEvent?

Wow, sorry for the monologue. My thoughts just kinda spilled right out there. Anyway, if someone wants to comment on the NetObject vs NetEvent for this application, I would appreciate it. Also the question at the end could use an answer :)

#10 "The netobject is inside the Simobject." I'm not sure what you mean by this. Can you elaborate?

I think you might be making this into a much more complicated situation than it really needs to be.

#11 When I declared the Manager SimObject, I declare a NetObject member field in the SimObject. I realize this may be overcomplicating it already, but I didn't understand another way to do it until I accidentally found the PathManager. Instead of just saying "that's too complicated," it would be more helpful to give another sentence or two offering an alternative approach, a reference to a useful class or document, or an explanation of why it's complicated. I'm not trying to put you down or anything, I respect you, and I appreciate your help. I'm just trying to improve the quality of the forums as a whole, as they are good documentation as long as they're thought of that way.