T2D as a DLL?

#1 It would be easier to integrate Python into T2D than to completely rewrite the core mentality of TGE/T2D into a dynamic link library.

#2 Would it require a total rewrite? :O

#3 It wasn't designed to be a library. You're thinking of a complete architecture restructuring. TAP is a game engine, not a library. It's akin to recompiling Word or AutoCAD or 3D Studio Max as a DLL. Creating a DLL interface to TGE would be a completely different matter, though--much like utilizing DLL interfaces to office products. But you'd have to know the engine intimately to get the right hooks. Plus, I'd imagine you'd have to do a huge amount of consolidation between the client and server code...

#4 Thanks for the advice. :)

#5 DLL's and EXE's are nearly identical except for file extensions.
EXE's have a int main(), but DLL's can have those as well, it just doesn't get executed when you double click the dll because the way Windows OS works.

You can export symbols from an exe just like you do a DLL, so file extension means nothing.

You could create a series of wrapper functions and export them (using declespec, or extern c etc..), and then load the module in Python, however I don't think this will give you the results you desire, I think your looking more so you can use Python as the scripting language instead of torque script, that process is completely different, you would have to look at implementing Python in a C application as a scripting language, and then implementing all the Class and function export system they have for Torquescript for python, which would not be fun.

-jeremy

#6 Josh Ritter's MMORPG Minions of Mirth (done in Torque) is driven by Python, might want to look him up.

#7 @Jeremy: Everything that you said is correct, however, David's point is that Torque is not designed to be used as an API (which would be what the DLL's would be providing), but is designed to be used as a fully functional networked simulation.

Sure, you could compile TGE as a DLL, but that wouldn't give you any benefit in trying to use very small portions of the engine as a library.

#8 Ideally, I want to use Python as the main program with calls to T2D and T2D events to be handled by Python. As mentioned, Josh Ritter (who hasn't answered any of my emails) did it with TGE, so it's possible. I just don't know how much work it requires to handle it that way.

As a short term measure, I'm embedding Python into T2D, but eventually I want Python controlling everything and T2D as just another module.

#9 Hmm, sounds like a very cool endeavor, though I really would support what was said, it makes more sense when making a game to use python inside of T2D, instead of vice versa, considering T2D is a game engine built on a gaming platform.

btw Josh is a very busy guy (the mmo is looking great).

#10 I'm working with T2D as a DLL -- it was the only reasonable way I could get it to communicate with Smalltalk. Fortunately, Torque's scripting system means that you don't need to expose much of the API; the functions I needed to write and export ending up being similar to the TGEPython code (which I used for guidance).

I'm not done yet, however, and I recently encountered one major caveat. Currently, I start the engine from Smalltalk, basically by calling T2D's main method. To prevent Smalltalk from hanging until the engine is stopped, I had to call it from a new thread. Any subsequent calls to or from Smalltalk were made from the main (VM) thread. I knew that T2D wasn't thread-safe, but everything seemed to be okay until I tried to create and register an image map from Smalltalk. This triggers OpenGL calls to register the texture, and a nasty exception occurs (it works, however, if I use the D3D wrapper).

So, despite my best efforts to avoid it, I'm afraid I'll have to control the game loop directly from Smalltalk, so as to avoid needing more than one native thread. T2D's initialization/main loop code isn't well factored, so that means more copying and pasting than I'd like.

To end on a positive note, the change needed to simply compile T2D as a DLL was trivial. I'm using TBE (which uses GCC), and all I needed to do was add the -shared flag to a make file. Of course, I also changed the file extension defined by $(EXT.EXE).

#11 T2D and TGE will work fine as a DLL with Python. I'm doing it now ;)

I'll be releasing a version of Josh's pytorque (he's cleared it, obviously) at some point when I'm able to de-MoM it and clean it up as a resource.

Caveat: It doesn't work well on Linux at this point ... Embedded is trivial, but TGE, Linux, and shared library is a bad combo at this point ;)

#12 Drop me a hint of how to do it, or send me some source code so we can team up. :) I'm in a hurry. :P

#13 Success! As I described above, I moved the code for initialization, shutdown, and running an iteration of the game loop into separate functions that could be called externally. The initialization and shutdown functions worked as expected, but the main loop did not. A game would run, sort of, but T2D was completely oblivious to user input. I realized that, because Smalltalk and T2D were now sharing the same thread, they were also sharing the Windows message queue. Argh! As a result, I ended up learning more about low-level Windows programming than anticipated. The solution was to give Smalltalk sole responsibility for managing the message queue (though it often delegates to T2D's "WindowProc" and "OurDispatchMessages").

It's now possible to register an image map via Smalltalk without raising an exception. I'm enthused by my progress, though I'm disappointed that I had to make more changes than expected to the engine (and Windows-specific ones at that). The amount of code I've had to write or modify is still modest, however.

I wonder how my approach compares to the one taken by Josh and Jeremy.

#14 That sounds really cool! Could you post some specifics as to what you changed? I'd be very interested to see it.

#15 Would it be possible to do something similar to turn T2D into a browser plugin? Would be cool to do some web-based games like what Popcap does only using T2D as a foundation.

#16 My work isn't ready to be posted as a resource yet, but you can download my engine modifications. If you set up T2D to be compiled under TBE, and unzip my file in Torque2D\SDK, you should be able to recreate the DLL. I included a diff file to make it easy to study my changes. (I was humbled to see how short it is! Where did all the time go? My excuses: I hadn't used C++ for years; I was unfamiliar with T2D, GCC, makefiles, and low-level Windows programming; I had to extend my knowledge of how Dolphin Smalltalk interacts with C++ and Windows. Also, while Eclipse is great for Java programming, the plugin for C++ is of limited use for browsing.)

All the exported DLL functions are in dllInterface.cc. There is no error handling yet; I plan to add some later. Also, a couple more functions need to be exported (e.g., for accessing array fields). Some helper functions were added to winWindow.cc and main.cc, and the message handling code in Platform::process() was commented out.

The file does not include any Smalltalk code. Here's a snippet from my workspace:

KernelLibrary default setCurrentDirectory: 'C:\Torque2D\SDK\example'.
lib := Torque2DLibrary default.
hwnd := lib startEngineWithArgs: '-mod spacescroller'.
t2d := T2DView fromHandle: hwnd.

Yes, T2D has an annoying dependency on the working directory (that could be changed by modifying Platform::getWorkingDirectory() in winFileio.cc).

Dolphin Smalltalk has the cool ability to create a View object for a window that was created externally. And then you can tell the view that it should use Dolphin's window procedure for processing messages.

T2DView is currently responsible for running the game loop (it repeatedly calls processEvents()). All messages except WM_CLOSE and WM_DESTROY are passed to T2D by calling t2dWindowProc(). As each message is dispatched to T2D, postGameEvents() is called. When a T2DView is closed, stopEngine() is called.

My explanation is probably as clear as mud, but it will have to do for now. ;-) Despite a couple of petty gripes about how T2D could be factored better, I'm impressed by how few changes were needed to use the engine as a DLL.

#17 Wow, Aaron, thanks very much for this information and your files! It's great that you are willing to share your work. I'll certainly study them. Please keep us updated of your progress.

#18 I've already benefited from the information on these forums (especially the TBE and TGEPython related stuff), so it's only fair that I share what I've done. I don't expect to do much more on the engine side. Most of my attention is now focused on making it as easy as possible to use T2D with Smalltalk. The DLL interface is language-agnostic; it could be used with anything. The downside is that it doesn't integrate as nicely with Python as TGEPython or its offshoots. For example, TGEPython allows you to create Python objects that map nicely to SimObjects; the attributes of Python objects map to SimObject fields, and Python method functions map to console methods. That's a neat trick that can't be copied in Smalltalk. Well, I could override #doesNotUnderstand:, but that's ugly, inefficient, and error-prone.

I'm re-implementing the tutorial in Smalltalk. I've just reached the "Collision" section, and everything works so far. But it ain't pretty. For example,

%projectile = new fxStaticSprite2D() { scenegraph = t2dSceneGraph; };
%projectile.setPosition( $player.getPosition() );

becomes

projectile := library newInstanceOf: 'fxStaticSprite2D'.
projectile setField: 'scenegraph' to: 't2dSceneGraph'.
projectile registerObject.
projectile execute: 'setPosition' with: (player execute: 'getPosition').

And that's using some methods built on top of the raw DLL interface. I plan to autogenerate some wrapper classes so that I can write instead:

projectile := FxStaticSprite2D scenegraph: 't2dSceneGraph'.
projectile setPosition: player getPosition.

I suppose it's time I created a .plan. ;-)

#19

Quote:The downside is that it doesn't integrate as nicely with Python as TGEPython or its offshoots. For example, TGEPython allows you to create Python objects that map nicely to SimObjects; the attributes of Python objects map to SimObject fields, and Python method functions map to console methods. That's a neat trick that can't be copied in Smalltalk.

TGEPython is embedded, which might make a difference. I'm not sure, as I haven't started working on this fully yet. I'll definitely be following your progress and will see what I can get working in the way of using your DLL code applied to Python. Let us know when that .plan is posted. ;)