Game Development Community

Torque networking, applied to VOIP

by Anthony Lovell · in Torque Game Engine · 02/04/2004 (9:18 am) · 92 replies

I have made some solid progress at adding VOIP into Torque (I can sample the microphone and compress it using Speex... code should work on Mac and Windows, and I could explore Unix). I intend to eventually share my work with the community when I have it ready, but now I need to learn much more about Torque -- primarily, its networking gestalt.

For my first implementation, I would like to just use Torque's unreliable UDP platformNet to send streaming voice packets through the server to only those other clients who are near enough to hear the speaker (a simple distance limit). Then, on each of those clients, I'd like them to hear the sound emanate from the position of the speaking player.

I had been thinking that my means of coding this was to be a direct use of Net::sendto() but another idea occurs to me:

Is the proper way to do this to add the mouth of the Player as part of the data state that the network layer is supposed to disseminate to other clients, much like his orientation in the world? In such a model, a non-talking player would have no state to transmit to the net, but when my microphone layer hands me some compressed audio, I'd hand it to my local Player instance's mouth, and trust that the neat differencing logic could then realize that it needs to go out to the net. If this is the approach, would I have to take special measures to ensure you could hear people behind you (as they are out of camera scope -- I take it this is different than setting scopeAlways, as the distance is still pertinent)?

Also, on the receive side, would the key be to add an AudioEmitter (or a subclass of one) to the mouth of the speaker? Is there any existing high-level support for audio which is provided on a frame-by-frame basis, and not from a soundfile resource?

Thanks in advance for any counsel you can provide.

tone
NextNext Thread
PrevPrevious Thread
More from Torque Game Engine
Page«First 1 2 3 4 5 Last »

Associate Kyle Carter
#21
03/16/2004 (9:55 am)
Ok, one minor problem here - if you're running a local server with a client and server in the same process, getClientGroup() will always return a valid result!

You should really add some printfs in there to show where it's sending stuff.

Torque Owner Duncan Gray
#22
12/01/2004 (10:52 am)
Anthony, how is this project going?

Torque Owner Anthony Lovell
#23
12/01/2004 (11:42 am)
I aborted it. Frankly, every time I run up against Torque, it kicks my ass.

I had it working to the point that I was sampling speech and compressing it, but had no idea how to network in Torque. I would need a partner.

Sadly, I tossed my work months ago, as I feared I would not recognize it anymore.

tone

Torque Owner Edward Gardner
#24
12/02/2004 (9:10 am)
I hadn't realized you had been workign on this before, looks as though you made some progress.

Torque 3D Owner Robert Blanchet Jr.
#25
12/02/2004 (9:14 am)
Never toss your work, even if you think what you wrote is "crap" or whatever. Always something useful to someone, possibly even yourself.

Torque Owner Brett Fattori
#26
12/02/2004 (9:15 am)
Anthony, if you'd like to work with someone, drop me an email at brettf AT renderengine DOT com. I'll see what I can do to get the tech working as you'd like. I've spent a lot of time, recently, working with the network and it's really not that difficult. I've been using events to send data that most would usually try to pack/unpack. It works quite well...

So, gimme a holler and let's see if we can't get this working like you want it to!

- Brett

Torque Owner Roger Delgado
#27
01/21/2005 (1:36 am)
Erm guys, I found this website phonesnd.com, a free P2P VoIP with sourcecode. I'm not a hardcore C++ programmer but maybe it can be integrated in torque(?)

Torque Owner Anthony Lovell
#28
02/07/2005 (10:59 am)
I'd enjoy working on this with you, Brett, if you'd email me please.
I tried emailing you at the profile address you provided, but wonder if it is stale?

tone

Torque Owner Sumner Mccarty
#29
02/15/2005 (11:10 am)
Greetings, Anthony... Roger,etc...

As VoIP is part of my current(and first Torque) project, I'm very interested in this progress...

I am a hardcore C++ programmer and don't see why the three of us couldn't easily figure out how to implement phonesnd lib or another P2P if necessary into Torque.

Please contact me asap at sumdj@pcdj.com or at sumner_mccarty@hotmail.com

Cheers!
:)
Sumner

Torque Owner Anthony Lovell
#30
07/05/2005 (8:30 am)
I am once again working on this (anew).

I am nearly at the point where some expert outside help would be a good thing.

My basis has been:

1.4 HEAD
Speex 1.1.10
PortAudio v18.1

At the moment, I am working on WIndows XP only, but the above choices should make the Mac work fairly easily after the fact. I am going for a model that will easily allow other uses to be made of the vox input (recording or speech recognition). That is, I don't want to bolt it too tightly to VOIP.

I have audio coming from the mike on a PTT model.
I will need help with streaming audio output and networking.

I intend to post my code on a CVS server soon so that others may ably assist.

tone

Torque Owner Anthony Lovell
#31
07/07/2005 (9:56 am)
I am having trouble. Having read http://www.garagegames.com/docs/tge/engine/classNetEvent.php I find FAQs are just not addressed.

What part of the source code shows me where the server receives a NetEvent and gets to decide which client(s) to send it to? Assuming there IS such a place (so that an event need not be sent to every client), does the server need to copy the event before sending to each interested client, or can it merely hand the same single copy to each and call post()?

Does NetEvent::process() get called when the event arrives in the server, or does this only get called from the clients who receive the event?

Does NetEvent::notifyDelivered() get called for unguaranteed events? When it is called, I presume it is called in the SENDING client?

This will help me get further, but I really wish the answers were within http://www.garagegames.com/docs/tge/engine/classNetEvent.php

tone

Torque Owner Duncan Gray
#32
07/08/2005 (1:42 pm)
I don't know if this helps but in game/main.cc you will find
while(Game->isRunning())
   {
      PROFILE_START(MainLoop);
      Game->journalProcess();
      Net::process();      // read in all events

Net::process(); is in Winnet.cc etc and that function ends with Game->postEvent(receiveEvent);

I have not delved beyond that yet.

Also, those people who offered to help in this thread may not be currently linked for updates from this thread because it's been a while since you were active.

Bottom line is they may not be aware you are working on this again. Best you email them directly and let them know, I'm sure they can help.

Torque Owner Anthony Lovell
#33
07/08/2005 (2:59 pm)
Hi Duncan...

I got some help from Ben Garney ... and a loose promise that I can bother him again as long as I am not wasting his time TOO MUCH. The IRC chat is a great resource as well.

I also tried to email Brett Fattori (as he appears to have manifest coding know-how), but no response.

tone

Torque Owner Michael-Paul Jakovlevski
#34
07/08/2005 (5:53 pm)
I can answer some of your questions. As far as the process function it seems to be called only on the recieving side. this can be either the client or server. You can broadcast the packets in the process funtion on the server. here is how i would do it. 
void VoxDataEvent::process(NetConnection *con)
{
	GameConnection *SourceCon = (GameConnection*)con;
	if(SourceCon->isServerConnection()) //we are on the client side.
	{
		//play sound
	}
	else //server side
	{
		ShapeBase *SourceCamera = SourceCon->getCameraObject();
		if(!ClientCamera)
			return;
		Point3F SourcePos = SourceCon->getTransform().getPosition();
		for(SimSetIterator it(Sim::ClientGroup);*it;it++)
		{
			GameConnection *gc = dynamic_cast<GameConnection*>(*it);
			if(!gc || gc == SourceCon)
				continue;
			GameBase ClientCamera = gc->getCameraObject();
			Point3F ClientPos = ClientCamera.getTransform()->getPosition();
			if(ClientPos - SourcePos).lenSquared() < gMaxSpeechRadius * gMaxSpeechRadius)
			{
				gc->postNetEvent(this); //NetEvent uses Reference tracking so should it's safe to do this
			}
		}
		
				
	}
}

Torque Owner Anthony Lovell
#35
07/12/2005 (7:21 am)
Thanks for your help -- I have my code largely framed by your suggestion (with syntax changes perhaps necessitated by 1.4), but there are still issues... perhaps owing to a bug in the networking layer that renders it intolerant of the logic you provide.

The issue, I fear, is that the event gets deleted in spite of the fact that it has a non-zero refCount, as found in this part of netEvent.cc:

if(unguaranteedPhase)
      {
         evt->process(this); 
         delete evt; // this is getting called even tho the gc->postNetEvent(this); in the code you supplied has driven evt->refCount > 0
         if(mErrorBuffer[0])
            return;
         continue;
      }

I'm puzzled that I find no place within the existing engine code where the server explicitly relays NetEvents (in C++) to other connected clients. Will this code perhaps start working if I alter the code to 


if(unguaranteedPhase)
      {
         evt->process(this); 
         if (!evt->mRefCount) // if no one else needs the event, delete it
             delete evt; 
         if(mErrorBuffer[0])
            return;
         continue;
      }

edit: this seems to fix the problem fine, and I hope it did so without causing memory leaks. I wonder is this just that no one has asked TGE to relay Unguaranteed NetEvents around from one client to another? The capability would seem fundamental to me!

tone

Associate Kyle Carter
#36
07/12/2005 (10:06 am)
People do all the time. But usually (since the NetEvent contains context-specific information) they just create a new one and relay that, rather than reposting the same one.

Hmm.

I think your fix is good. Are there any other places that things might deleted extraneously?

Bug #141.

Torque Owner Anthony Lovell
#37
07/12/2005 (10:41 am)
A quick check reveals only one other delete call in that file, and it may also want the same check (can't say I grok the intent of the mSendingEvents bool).

I hear what you are saying re: state data possibly being out of whack. I think I will keep what I have (for now) and think some on possible ramifications. My intent (possibly naive... I'm new!) was to preserve data on whose client sent the speech (mSourceId? --- the crib Michael-Paul supplied is largely working in terms of allowing me to calculate distance between speaker and listener).

tone

Associate Kyle Carter
#38
07/12/2005 (10:51 am)
Ok, I added several decRef()s in place of delete, and also an assert in the destructor for NetEvent if the ref count isn't zero. This is all in SVN, look for the code when RC2 comes out for TGE 1.4.

Torque Owner Anthony Lovell
#39
07/12/2005 (11:28 am)
Ben ... I'm not 100% sure that decRef is the robust fix. The refCount at the moment process() is called in the server is generally == 0, not 1. I worry that in cases where it is > 0 that decrementing it at that point is not appropriate. It might bear a little broader examination (although I don't want to voice this too loudly in case you've given it a fair review).

tone

Torque Owner Michael-Paul Jakovlevski
#40
07/12/2005 (4:47 pm)
Didn't see that delete. i found this this one 
if(!mSendingEvents)
   {
      delete theEvent;
      return false;
   }


but mSendingEvents is set to true for both client & server for GameConnections.
if you change this code to decref it will cause the refcount to go to -1 and decref will not delete it. so do this:
if(!mSendingEvents)
   {
       theEvent.addRef(); //sets it to 1 if new
       theEvent.decRef(); //deletes if new

      return false;
   }

(Oh the joys of refcounting)

The reason I would reuse the event is that it handling a lot of these packets and it cut down on exessive copyng. I would definatly test this out for performance. it might slow down the server for all i know. Expecially with multiple people talking. it might be necessary to do a range check at larger intervals, storing the results in a array. it would cut down on how many times that loop is called.
Page«First 1 2 3 4 5 Last »