T3D Audio Loopback Demo
by Demolishun · 12/19/2012 (3:16 am) · 6 comments
I finally got it working how I want for the most part. This is a demo of capturing loopback audio in Windows 7 (or Vista) and later. Once the data is captured from the output device it is run through a FFT available here called KISS FFT. KISS FFT is a BSD licensed library for doing frequency analysis on data.
The demo video to prove it happened:
Stats:
The next steps:
BTW, the voice sound quality of my description is pretty bad. It sounded okay on headphones, but pretty bad on a boombox used as speakers. So don't sweat it if you can't here my description in the video.
Edit:
I didn't want to start a new blog, but I found a nice addition to this. It won't necessarily be used with this, but it could complement this feature. I am going to look at incorporating MIDI into T3D/T2D as well. I found a nice library called: PortMedia/PortAudio that looks like it is MIT licensed. It has more than just MIDI. Everything is cross platform as well. I have always wanted to have music that responds to user actions and affects the mood of the game through visuals. Having the ability to track external music is part of that. Playing midi allows individual instruments to be turned on and off so songs can adapt to the gameplay. It might even be possible to track an external song and play midi that complements that music. Truly individualized gameplay.
The demo video to prove it happened:
Stats:
- Runs every 50mS in a separate thread.
- Data can be grabbed using script commands.
- The demo runs a schedule every 100mS to update the widgets on the main menu.
- I still have no idea why I used the GraphCtrl except I tweaked it until it looked cool.
The next steps:
- Clean up the code.
- Make it available as a resource.
- Port it T2D as I am sure once you see the video you will want this in T2D. I do!
- Enlist other people into providing OSX and Linux ports for the loopback capture code.
- Entice someone to write a GUI widget that allows proper output of a frequency graph as a 2D widget/hud.
BTW, the voice sound quality of my description is pretty bad. It sounded okay on headphones, but pretty bad on a boombox used as speakers. So don't sweat it if you can't here my description in the video.
Edit:
I didn't want to start a new blog, but I found a nice addition to this. It won't necessarily be used with this, but it could complement this feature. I am going to look at incorporating MIDI into T3D/T2D as well. I found a nice library called: PortMedia/PortAudio that looks like it is MIT licensed. It has more than just MIDI. Everything is cross platform as well. I have always wanted to have music that responds to user actions and affects the mood of the game through visuals. Having the ability to track external music is part of that. Playing midi allows individual instruments to be turned on and off so songs can adapt to the gameplay. It might even be possible to track an external song and play midi that complements that music. Truly individualized gameplay.
About the author
I love programming, I love programming things that go click, whirr, boom. For organized T3D Links visit: http://demolishun.com/?page_id=67
#2
I figure it is because of holidays. I gasp to think people have lives outside game programming...I know I will try and block that out.
I got to talking with a friend of mine about the whole music tracking thing and deciphering notes in realtime would be very hard. You could process a song and then replay, but it would still be hard to pick out chords and the like. I am going to pursue playing midi though. I have some ideas on this and I want to explore some musical interaction using T2D and eventually T3D. Seeing how my kids respond to simple music games on a phone I want to see what would happen with 4 kids on a PC in a living room.
12/21/2012 (4:02 am)
@Lukas,I figure it is because of holidays. I gasp to think people have lives outside game programming...I know I will try and block that out.
I got to talking with a friend of mine about the whole music tracking thing and deciphering notes in realtime would be very hard. You could process a song and then replay, but it would still be hard to pick out chords and the like. I am going to pursue playing midi though. I have some ideas on this and I want to explore some musical interaction using T2D and eventually T3D. Seeing how my kids respond to simple music games on a phone I want to see what would happen with 4 kids on a PC in a living room.
#3
12/25/2012 (9:33 pm)
Pretty interesting but I don't see any real application for game development for this. Maybe you can explain a bit more on what exactly this would be useful for a game developer.
#4
Also, Torque is not limited to games anymore.
12/26/2012 (2:37 am)
Personalized gaming experience. The player chooses the music, the game reacts to the music. That was the original intent. I am sure there will be many more uses. Think of it as a tool that might be able to solve a problem.Also, Torque is not limited to games anymore.
#5
02/07/2013 (3:28 pm)
Torque can be more than just a game engine, with all this virtual reality stuff happening it can also be a VR platform for applications. I can see this being used to make music and djing apps. 
Torque Owner Lukas Joergensen
WinterLeaf Entertainment
But this is a really cool feature, just have to sit around and wait for when I see "iTunes made with Torque 3D" in an advertisement! ^^
Or the new Guitar Hero lol, would totally be possible with this.