ITunes / T2D Interaction
by Joshua "RegularX" Birk · in Torque Game Builder · 05/08/2005 (3:30 pm) · 16 replies
My first couple of projects are centering around combining music tightly with gameplay, a la Rez or Lumines (though oddly, I haven't played either...)
I've got a couple snags, some things I could make friendlier and I need to smooth out some performance hiccups which are completely of my own doing, but after a lot of finagling I've got an XMLRPC server running from an iTunes visualizer. This allows me to access the audio data from any XMLRPC client, which I've shoved into T2D.
Screenshot
There's nothing remotely playable yet, but I'm going to try and hammer that out this coming week. But today I finally fixed the last bug which was shutting down the server (I had T2D creating multiple clients ... eeeek!) I've got one simple concept laid out - kinda a space invaders meets missle command type gameplay which I think I can get working in fairly short order.
A couple of times this weekend I thought it might be a non-starter as it almost seemed like either the XMLRPC lib or Torque itself might not be friendly with the task, but it just turned out to be my lousy coding :) I still have some fundamental problems, like I said, before I can claim victory - but this was probably the biggest hurdle to proving it possible.
I've got a couple snags, some things I could make friendlier and I need to smooth out some performance hiccups which are completely of my own doing, but after a lot of finagling I've got an XMLRPC server running from an iTunes visualizer. This allows me to access the audio data from any XMLRPC client, which I've shoved into T2D.
Screenshot
There's nothing remotely playable yet, but I'm going to try and hammer that out this coming week. But today I finally fixed the last bug which was shutting down the server (I had T2D creating multiple clients ... eeeek!) I've got one simple concept laid out - kinda a space invaders meets missle command type gameplay which I think I can get working in fairly short order.
A couple of times this weekend I thought it might be a non-starter as it almost seemed like either the XMLRPC lib or Torque itself might not be friendly with the task, but it just turned out to be my lousy coding :) I still have some fundamental problems, like I said, before I can claim victory - but this was probably the biggest hurdle to proving it possible.
About the author
#2
The plan is to make it playable if iTunes isn't in the background, though, just you know, not as much fun.
05/08/2005 (4:41 pm)
Well on OSX nobody would have to download anything, and iTunes is free for Windows.The plan is to make it playable if iTunes isn't in the background, though, just you know, not as much fun.
#3
05/08/2005 (10:38 pm)
Coolness :) 
#4
05/09/2005 (1:33 am)
Joshua, I'd be interested to see how you do this for Mac since I've been doing a lot of iTunes/Windows integration. Check my last .plan.
#5
So instead I used the XMLRPC++ library and the VizKit visualizer example, which are both moderately lightweight and cross-platform solutions, and hamstrung them together. So whenever my visualizer renders a frame, it also gives the RPC server a kick. Then it was just a matter of including the library in my Torque project and creating an datablock which acted like an XMLRPC client with some handy console functions.
Then it's just matter of declaring a rgxAudioDatablock in the Torquescript. I'm using Langley's handy explode function to seperate out the spectrum data, which is what you see in the game's console window.
Biggest problem right now - game hangs when the datablock is waiting a response from the server. This ranges from annoying to crashworthy. I need to seperate out and possibly properly thread that function so that it's not a bottleneck.
So right now I've got it plausible, but not playable. Also, the visualizer must be operating for the server to run, which isn't the worst thing but will require some user herding to compensate - or figure out a non-visualizer plugin angle ... but sadly there's even less documentation for that.
I can try and bundle up some code if you want to peek Pat - although I'd suggest danger-sensitive glasses upon viewing. Considering this has taken me a few weeks, it's actually sadly straightforward ... I think I've removed more code than I've added at this point.
05/09/2005 (5:40 am)
Well the "common" way to do iTunes interapp communication on OSX is I guess AppleScript. Course, I'm not sure if that opens the kind of exposure I want - like actual audio data, or if it's just control functions like pause, play, etc.So instead I used the XMLRPC++ library and the VizKit visualizer example, which are both moderately lightweight and cross-platform solutions, and hamstrung them together. So whenever my visualizer renders a frame, it also gives the RPC server a kick. Then it was just a matter of including the library in my Torque project and creating an datablock which acted like an XMLRPC client with some handy console functions.
Then it's just matter of declaring a rgxAudioDatablock in the Torquescript. I'm using Langley's handy explode function to seperate out the spectrum data, which is what you see in the game's console window.
Biggest problem right now - game hangs when the datablock is waiting a response from the server. This ranges from annoying to crashworthy. I need to seperate out and possibly properly thread that function so that it's not a bottleneck.
So right now I've got it plausible, but not playable. Also, the visualizer must be operating for the server to run, which isn't the worst thing but will require some user herding to compensate - or figure out a non-visualizer plugin angle ... but sadly there's even less documentation for that.
I can try and bundle up some code if you want to peek Pat - although I'd suggest danger-sensitive glasses upon viewing. Considering this has taken me a few weeks, it's actually sadly straightforward ... I think I've removed more code than I've added at this point.
#6
I've got a working quasi-Tempest clone. The green "vertex" is controlled by the audio data. The red lines are what you are trying to keep from getting to the surface. The play can rotate around the edge of the tempest and shoot away the red lines.
The game is also properly spawned from the visualizer itself now. I'm not sure the game will ever be seperate from iTunes now that I play with it more. Don't really see the point.
Much thanks to Hans for posting the beam code ... which got me thinking about this.
Will work on some basic scoring and then a lot of beautification next.
05/15/2005 (4:17 pm)
webcosa.com/images/UTC.jpgI've got a working quasi-Tempest clone. The green "vertex" is controlled by the audio data. The red lines are what you are trying to keep from getting to the surface. The play can rotate around the edge of the tempest and shoot away the red lines.
The game is also properly spawned from the visualizer itself now. I'm not sure the game will ever be seperate from iTunes now that I play with it more. Don't really see the point.
Much thanks to Hans for posting the beam code ... which got me thinking about this.
Will work on some basic scoring and then a lot of beautification next.
#7
cathodetan.blogspot.com/2005/05/dev-day-diary-gettin-purty.html
Mostly have worked on making things a little prettier, will get back into the guts of the audio interaction soon.
05/17/2005 (5:38 pm)
Updated pic here:cathodetan.blogspot.com/2005/05/dev-day-diary-gettin-purty.html
Mostly have worked on making things a little prettier, will get back into the guts of the audio interaction soon.
#8
The big step here would be the barely visible purplish flecks coming from the middle. They are single pixel white images that I'm resetting the colors to dynamically. Next step is to make more complicated images procedurally.
06/08/2005 (5:48 pm)
The big step here would be the barely visible purplish flecks coming from the middle. They are single pixel white images that I'm resetting the colors to dynamically. Next step is to make more complicated images procedurally.
#9
06/08/2005 (6:29 pm)
Cool. Way to keep at this, Josh. I love this stuff, as anyone at GG will tell you I always have my iTunes up on my 2nd monitor with the visualization going and can often be caught looking lost in it. Please keep posting your progress.
#10
If a game said anywhere on it, that it could use iTunes / QuickTime, I just won't install it or buy it. I know a lot of other that think the same way. I am not saying this functionality isn't cool, but in theory it could hurt sales of your product.
06/08/2005 (8:05 pm)
Just a little side not on this functionality. Although its neat, there are two types of people, those that love iTunes and those that hate it. I am a windows user for example, and QuickTime / iTunes makes me extremely angry. If I install it, it creates a little icon on my task bar that is loaded every time I boot my computer. Granted I can disable it in the registry, but the next time I run QuickTime it will come back. In addition, on Windows they want you to buy QuickTime to use all its functionality (when i already have two free media players, Winamp and Windows Media Player). And if all that wasn't enough the really make a Windows user unhappy, it has ads that in the main window when you load it.If a game said anywhere on it, that it could use iTunes / QuickTime, I just won't install it or buy it. I know a lot of other that think the same way. I am not saying this functionality isn't cool, but in theory it could hurt sales of your product.
#11
06/08/2005 (8:23 pm)
Well, that's the handiness of the XMLRPC setup. The audio input doesn't necessarily have to be iTunes.
#12
Right now, there's a certain handiness in the fact that the Apple's viz demo code, VizKit and the XMLRPC lib that I'm using are all in some way open source, meaning that I can right the visualizer part of this, which is actually a server, and not worry about license. The Torque side needs some new client code to A) be properly closed source and B) be friendlier to iTunes.
So for now my focus is on Mac and iTunes. It's definately a culture which would probably embrace a music-visual-game combination. It's also handily the only environment I have right now. My main goal is to make it fluid and lightweight enough that it can be enjoyed by 1-2 people on either a laptop or perhaps even a Mac hooked to a TV.
Beyond that I could see adapting it to work with WinAmp, which I acknolwedge is much more prevalent outside of the House of Jobs, perhaps via a Media PC platform (probably WMP) ... which is evolving, or maybe even as a standalone MP3 player. But that's all down the line, once I get the first proof of concept running. However, the current setup is very client-server, disconnected from each other and would make it easy to adapt the game to different incoming sources in the future. It's just a matter of formatting and reading the data.
So yeah, I get the whole "watch where you tie your horse" issue with marketing ... but I'm not really worried about marketing at this point. I haven't even decided if I would charge for this on release.
I'm not sure if the sockets code in the OSX version of T2D is running right, so I'm focusing back on the visualizer aspect for now ... which means writing my own procedural texture framework. Or something similar. I can't promise Llamasoft, particularly when I'd like to use this setup for non-audio style games, but it is lots of fun seeing it start to click.
06/09/2005 (7:59 am)
OK, now that I have a bit more time, let me not put that so cryptically.Right now, there's a certain handiness in the fact that the Apple's viz demo code, VizKit and the XMLRPC lib that I'm using are all in some way open source, meaning that I can right the visualizer part of this, which is actually a server, and not worry about license. The Torque side needs some new client code to A) be properly closed source and B) be friendlier to iTunes.
So for now my focus is on Mac and iTunes. It's definately a culture which would probably embrace a music-visual-game combination. It's also handily the only environment I have right now. My main goal is to make it fluid and lightweight enough that it can be enjoyed by 1-2 people on either a laptop or perhaps even a Mac hooked to a TV.
Beyond that I could see adapting it to work with WinAmp, which I acknolwedge is much more prevalent outside of the House of Jobs, perhaps via a Media PC platform (probably WMP) ... which is evolving, or maybe even as a standalone MP3 player. But that's all down the line, once I get the first proof of concept running. However, the current setup is very client-server, disconnected from each other and would make it easy to adapt the game to different incoming sources in the future. It's just a matter of formatting and reading the data.
So yeah, I get the whole "watch where you tie your horse" issue with marketing ... but I'm not really worried about marketing at this point. I haven't even decided if I would charge for this on release.
I'm not sure if the sockets code in the OSX version of T2D is running right, so I'm focusing back on the visualizer aspect for now ... which means writing my own procedural texture framework. Or something similar. I can't promise Llamasoft, particularly when I'd like to use this setup for non-audio style games, but it is lots of fun seeing it start to click.
#13
Otherwise there are a few bsd style libs around. Here's one: http://xmlrpc-c.sourceforge.net/features.php. Never used it...just spotted it on google. Says it passes the xml-rpc validator and the code snippets look simple enough.
Looks game looks fun...I'm an old Tempest fan. :)
Nigel
06/09/2005 (3:19 pm)
Josh, I noticed your comments about your LGPL XMLRPC library on your blog link. If you're linking dynamically you should be fine. Aren't the OpenAL libs LGPL as well?Otherwise there are a few bsd style libs around. Here's one: http://xmlrpc-c.sourceforge.net/features.php. Never used it...just spotted it on google. Says it passes the xml-rpc validator and the code snippets look simple enough.
Looks game looks fun...I'm an old Tempest fan. :)
Nigel
#14
I've played with the xmlrpc-c lib, and while it's good - it's not very cut and dry due to it's reliance on the Abyss server code - but I haven't ruled it out as a replacement by any means. My needs are pretty small, though, since I will only need about 2-3 remote functions, tops, and they'll all be returning fairly straightforward strings ... so I'm hoping just rolling my own request will do the trick. The Torque event setup might be exactly what I need to smooth out the interaction between the client and visualizer, but unfortunately that's the very part I am having problems with. But even if neither of those pan out, there are some sockets libs I should be use as well. Always a way ... always a way...
06/10/2005 (7:20 am)
You know, I looked into dynamically linking it instead and about popped a cranial nerve in the process. Unless there is something obvious I'm missing (entirely possible, being out of C/C++ for a loooooooong time) it would seem somewhat painful. I think there should be some kind of law that anything LPGL should be available as a library from the start :) But like I said, there might be something simple I'm overlooking.I've played with the xmlrpc-c lib, and while it's good - it's not very cut and dry due to it's reliance on the Abyss server code - but I haven't ruled it out as a replacement by any means. My needs are pretty small, though, since I will only need about 2-3 remote functions, tops, and they'll all be returning fairly straightforward strings ... so I'm hoping just rolling my own request will do the trick. The Torque event setup might be exactly what I need to smooth out the interaction between the client and visualizer, but unfortunately that's the very part I am having problems with. But even if neither of those pan out, there are some sockets libs I should be use as well. Always a way ... always a way...
#15
07/05/2005 (12:05 pm)
Looks interesting. XMLRPC++ looks like it uses the STL. Did that cause many conflicts when you were trying to intergate it with Torque?
#16
This weekend I went into the gameplay itself and realized I might want to drift away from the hardcore Tempest-style and might go for something more shooter orientated. I was trying various iterations, like having climbing red bars hurt when they hit the top, or block the ship from rotating and while it was mildly entertaining nothing was really snapping like I had hoped.
I also need to continue to tweak the interapp launching, as iTunes will occasionally throw an error about not being able to save to the music library and completely hangs the game. It's also stingy about being towards the front, covering the game if it's windowed. Naturally I'd like to have the game run within iTune's viz window, but that's like a whole other framework.
07/05/2005 (12:36 pm)
Seemingly none. All of the integration went pretty swimmingly, minus my own newbish problems with C++. Mind you, it's pretty straightforward connect, update var, report var. Sadly it looks like I'll have to roll my own as the T2D socket code seems busted for OSX and I can't figure out how to lib the current rpc code. But since it is just connect, update, report - I'm hoping that won't be too painful.This weekend I went into the gameplay itself and realized I might want to drift away from the hardcore Tempest-style and might go for something more shooter orientated. I was trying various iterations, like having climbing red bars hurt when they hit the top, or block the ship from rotating and while it was mildly entertaining nothing was really snapping like I had hoped.
I also need to continue to tweak the interapp launching, as iTunes will occasionally throw an error about not being able to save to the music library and completely hangs the game. It's also stingy about being towards the front, covering the game if it's windowed. Naturally I'd like to have the game run within iTune's viz window, but that's like a whole other framework.
Torque Owner Charlie Malbaurn