Torque networking, applied to VOIP
by Anthony Lovell · in Torque Game Engine · 02/04/2004 (9:18 am) · 92 replies
I have made some solid progress at adding VOIP into Torque (I can sample the microphone and compress it using Speex... code should work on Mac and Windows, and I could explore Unix). I intend to eventually share my work with the community when I have it ready, but now I need to learn much more about Torque -- primarily, its networking gestalt.
For my first implementation, I would like to just use Torque's unreliable UDP platformNet to send streaming voice packets through the server to only those other clients who are near enough to hear the speaker (a simple distance limit). Then, on each of those clients, I'd like them to hear the sound emanate from the position of the speaking player.
I had been thinking that my means of coding this was to be a direct use of Net::sendto() but another idea occurs to me:
Is the proper way to do this to add the mouth of the Player as part of the data state that the network layer is supposed to disseminate to other clients, much like his orientation in the world? In such a model, a non-talking player would have no state to transmit to the net, but when my microphone layer hands me some compressed audio, I'd hand it to my local Player instance's mouth, and trust that the neat differencing logic could then realize that it needs to go out to the net. If this is the approach, would I have to take special measures to ensure you could hear people behind you (as they are out of camera scope -- I take it this is different than setting scopeAlways, as the distance is still pertinent)?
Also, on the receive side, would the key be to add an AudioEmitter (or a subclass of one) to the mouth of the speaker? Is there any existing high-level support for audio which is provided on a frame-by-frame basis, and not from a soundfile resource?
Thanks in advance for any counsel you can provide.
tone
For my first implementation, I would like to just use Torque's unreliable UDP platformNet to send streaming voice packets through the server to only those other clients who are near enough to hear the speaker (a simple distance limit). Then, on each of those clients, I'd like them to hear the sound emanate from the position of the speaking player.
I had been thinking that my means of coding this was to be a direct use of Net::sendto() but another idea occurs to me:
Is the proper way to do this to add the mouth of the Player as part of the data state that the network layer is supposed to disseminate to other clients, much like his orientation in the world? In such a model, a non-talking player would have no state to transmit to the net, but when my microphone layer hands me some compressed audio, I'd hand it to my local Player instance's mouth, and trust that the neat differencing logic could then realize that it needs to go out to the net. If this is the approach, would I have to take special measures to ensure you could hear people behind you (as they are out of camera scope -- I take it this is different than setting scopeAlways, as the distance is still pertinent)?
Also, on the receive side, would the key be to add an AudioEmitter (or a subclass of one) to the mouth of the speaker? Is there any existing high-level support for audio which is provided on a frame-by-frame basis, and not from a soundfile resource?
Thanks in advance for any counsel you can provide.
tone
#82
I would love to work on this as well. However, I'm not a natural-born coder. As for a license and credit, personally, I would love to see this completed and released without charge to the Garage Games community. That way, you would have full use of it forever. And your name and credits would could stay in the resultant code resouces.
As a little background, I've been spawning external Skype sessions from Torque for my voice stuff, but an integrated solution would be ideal. Maybe your VOIP solution could be what many of us have been looking for.
If you think I might be a candidate for this, I really would love to work on it.
Thanks,
Aaron Ellis
07/05/2006 (9:29 am)
Anthony,I would love to work on this as well. However, I'm not a natural-born coder. As for a license and credit, personally, I would love to see this completed and released without charge to the Garage Games community. That way, you would have full use of it forever. And your name and credits would could stay in the resultant code resouces.
As a little background, I've been spawning external Skype sessions from Torque for my voice stuff, but an integrated solution would be ideal. Maybe your VOIP solution could be what many of us have been looking for.
If you think I might be a candidate for this, I really would love to work on it.
Thanks,
Aaron Ellis
#83
07/05/2006 (11:33 am)
@Aaron - We could also work together on this if you wish. Two heads are better than none (pun intended)
#84
If Anthony is willing, I think that would be great. But again, I'm hoping this can be released to the Torque community for free to TGE/TSE/TGB SDK licencees. IMO, something this important and far-reaching should be available to everyone. :)
Aaron E.
07/05/2006 (12:02 pm)
Jonathon,If Anthony is willing, I think that would be great. But again, I'm hoping this can be released to the Torque community for free to TGE/TSE/TGB SDK licencees. IMO, something this important and far-reaching should be available to everyone. :)
Aaron E.
#85
07/05/2006 (12:25 pm)
@Aaron - I have no issues releasing free resources to the community. I'll have to evaluate how much work is left to be done before I can commit to releasing it free. If there's quite a bit of work that will take us several months as a team, then I might not be able to commit.
#86
I see where you're coming from. Fortunately, from Anthony's posts, it sounds like his solution is already in a usable state. If that's the case, it might be possible to pull his engine changes into a working resource, provide documentation, and then release it. At that point, people can start using it right away.
Of course this is all academic at this point - until Anthony decides how to proceed.
In spite of my minor successes with Skype and Torque, I'm still hoping to find a more integrated VoIP system - something that will make life easier for my end users.
Aaron E.
07/05/2006 (1:29 pm)
@Jonathon,I see where you're coming from. Fortunately, from Anthony's posts, it sounds like his solution is already in a usable state. If that's the case, it might be possible to pull his engine changes into a working resource, provide documentation, and then release it. At that point, people can start using it right away.
Of course this is all academic at this point - until Anthony decides how to proceed.
In spite of my minor successes with Skype and Torque, I'm still hoping to find a more integrated VoIP system - something that will make life easier for my end users.
Aaron E.
#87
In essence, this is DONE for 3D chat.
I just have no idea how to present it simply as an addition to (say) starter.FPS
It's future roadmap, IMO, is:
1. finish and test the 2D channel-based tuning and propagation model
2. add mute lists to squelch troublesome players
3. allowing privileged users to adjust the input gain of people who have no idea how loud or soft they are would be nice
4. Automatic Gain Control would be nice, too
5. Ideally, the input gain should be done in hardware and not in software
I'd hand this over to anyone who will commit to showing a Fraps-recorded starter.FPS demo video on the site within 4 weeks and releasing the working code and scripts as a resource within 10 weeks.
You can email me at tone@dreadnoughtproject.org (click link found in challenge email you'll get immediately)
tone
07/05/2006 (2:48 pm)
The point is that I don't want to see this torn asunder and modified into 2+ forms that will enjoy uneven adoption before the first consumable version is available to all.In essence, this is DONE for 3D chat.
I just have no idea how to present it simply as an addition to (say) starter.FPS
It's future roadmap, IMO, is:
1. finish and test the 2D channel-based tuning and propagation model
2. add mute lists to squelch troublesome players
3. allowing privileged users to adjust the input gain of people who have no idea how loud or soft they are would be nice
4. Automatic Gain Control would be nice, too
5. Ideally, the input gain should be done in hardware and not in software
I'd hand this over to anyone who will commit to showing a Fraps-recorded starter.FPS demo video on the site within 4 weeks and releasing the working code and scripts as a resource within 10 weeks.
You can email me at tone@dreadnoughtproject.org (click link found in challenge email you'll get immediately)
tone
#88
From your list of features and the roadmap for this, it sounds exactly like what I would hope for in a 3D VoIP solution. Unfortunately, I can't guarantee that I will have it feature-complete in 4 weeks and documented/packaged in 10. Without guidance, it would take me a while to understand your changes and then attempt to extend them.
Hopefully, someone will read through this thread and take up the challenge.
Thank you,
Aaron E.
07/06/2006 (2:33 pm)
Anthony,From your list of features and the roadmap for this, it sounds exactly like what I would hope for in a 3D VoIP solution. Unfortunately, I can't guarantee that I will have it feature-complete in 4 weeks and documented/packaged in 10. Without guidance, it would take me a while to understand your changes and then attempt to extend them.
Hopefully, someone will read through this thread and take up the challenge.
Thank you,
Aaron E.
#89
With a deadline, especially that short, I just couldn't commit to this.
07/07/2006 (7:29 am)
I would have to agree with Aaron. I wanted this project for two main reasons; So the community could have a VoIP solution; So I could learn about an area (sound and VoIP) that I've yet to really touch on.With a deadline, especially that short, I just couldn't commit to this.
#90
07/07/2006 (7:46 am)
I'm building web-cam feature in to Torque (I have image capture and transmission working but current have problem with JPG compression). Your VOIP solution would go well with my solution. I too wish to release it as a resource, best of luck!
#91
07/07/2006 (7:55 am)
Nice paul, are you going to release that as a resource?
#92
Yes, but only a mini resource, enough to get you going. Unfortunately for now it's windows only.
pgmedia.awardspace.com/Site/TMessenger/screenshot.php
07/07/2006 (7:59 am)
Quote:Nice paul, are you going to release that as a resource?
Yes, but only a mini resource, enough to get you going. Unfortunately for now it's windows only.
pgmedia.awardspace.com/Site/TMessenger/screenshot.php
Torque 3D Owner Jonathon Stevens