Prev/Next Blog
by date

TGEA/TGB, GCC-XML, and Python

Name:	Prairie Games
Date Posted:	Nov 26, 2007
Rating:	4.0 out of 5
Public:	YES
Comments:	YES
RSS Feed:	or Subscribe with .
Profile Page:	View profile page for Prairie Games

Blog post

I recently spent a couple weeks evaluating technology options for our next game. It felt good to be in this mode... though, I think it felt better coming to a resolution. :)

We're going with Torque Game Engine Advanced. I also really want to use Torque Game Builder for the 2D and minigame elements. TGB is really, really advanced compared to the stock TGE/TGEA 2D game stuff. I think this will work out great. We've got TGB rendering using Direct3D9 and I should be able to get them glued together nicely with Python.

We've gathered a fair amount of experience using Torque and Python together. I've been satisfied hooking into the TorqueScript console using its "everything is a string" design. This gives us access to console objects and the ability to expose Python functions to the console system. It is pretty basic and has worked out pretty well...

While evaluating Ogre and Qt as a possible solutions for rendering/GUI, I checked out their Python wrappers: Python-Ogre and PyQT. They both use GCC-XML to parse the C++ code, generate an XML representation, and automatically generate binding code from the XML. This was really interesting... and a long story short: it works out great with Torque.

Here's the process:

1. GCC-XML is used to compile the Torque header files and generate a full XML representation of the namespaces, classes, enumerations, etc

2. I wrote a custom tool to take the C++ information generated by GCC-XML, filter it using pygccxml, and automatically generate a SIP interface file

3. SIP then generates the wrapper sources. Importantly, the wrapper code is in 100% C++, tight, and quite efficient. SIP generates very good bindings compared to tools like SWIG, Boost::Python, etc. It was written for PyQT and that rocks :)

There are some really, really big wins over the "everything is a string" method of the previous PyTorque bindings. The biggest thing is that it is now possible to derive new script classes directly from Torque C++ classes with full inheritance, virtual methods, variable access, enumerations, structures, etc... it is all accessible. There's quite a bit of new functionality... some of which is described in the example below. This is really powerul stuff!!!

I've been looking into C# a bit too... and it would definitely be possible to generate similar bindings using GCC-XML. I will probably look into this more once IronPython (Microsoft's Python implementation built on the DLR) is more mature.... that won't be for this game :)

I've got a bit more work to do on the bindings... want to get an integration pass done with TGEA + TGB... and then, WOOHOO!!!

-Josh Engebretson
President
Prairie Games, Inc
MMOWorkshop.com

Here's some example usage

#The pytgb.engine module is a (fast) Python C++ wrapper for TGB
#The module is automatically generated by a tool which leverages:
# GCC-XML: http://www.gccxml.org
# pygccxml: http://www.language-binding.net/pygccxml/pygccxml.html
# SIP: http://www.riverbankcomputing.co.uk/sip/index.php

from pytgb.engine import *
 
#This isn't your daddy's TorqueScript... 
#You can derive script classes directly from C++ classes with full inheritance
class MyBitmapButtonCtrl(GuiBitmapButtonCtrl):
    def __init__(self):
        #We can call into the C++ constructor!
        GuiBitmapButtonCtrl.__init__(self)
        
        #Fields can be set/read and feature automatic type conversion
        self.toolTip = "Click me!"
        self.visible = True
        
        #C++ Enumerations are fully supported
        self.buttonType = self.ButtonTypePush
        
        #Dynamic variables can be set and read 
        self.setDataField("myDynamicValue", "", "Wahwahweewah!")
        
        #Our own Python attributes can of course be added
        self.coolness = True

    #It is possible to override any virtual class method... defined in C++!
    def onAction(self):
        #We can call base classes implementation.
        self.parentClass.onAction(self) 
        print "I was clicked!"
        
    def inspectPostApply(self):
        self.parentClass.inspectPostApply(self)
        print "I've been applied!"
        
    def onAdd(self):
        self.parentClass.onAdd(self)
        print "I've been added!"

    #We can also add entirely new methods (and expose them to the console system... or not!)        
    def setCoolness(self, value):
        self.coolness = value

    def getCoolness(self):
        return self.coolness
        
#This behaves exactly like the IMPLEMENT_CONOBJECT macro C++ side.  
#It hooks our (dynamic) Python class into the Torque console system
#as a first class citizen!
IMPLEMENT_CONOBJECT(MyBitmapButtonCtrl)

#If you need to access methods from TorqueScript... 
#they can be exposed as ConsoleMethods (just like C++ side)
#Though, keep in mind that we can derive new Python classes from C++ classes.  
#So, you often wouldn't need to expose them for scripting purposes!
ConsoleMethod(MyBitmapButtonCtrl.setCoolness, "(boolean)", 3, 3)    
ConsoleMethod(MyBitmapButtonCtrl.getCoolness, "boolean ()", 2, 2)    


#Console functions can also be exposed
def HelloWorld():
    return "Hello World!"

#Again, just like C++ side
ConsoleFunction(HelloWorld, "()", 1, 1)

#Console funtions can also take arguments
def HelloWorldWithArgument(arg):
    return "Hello World with Argument: %s", arg
    
ConsoleFunction(HelloWorldWithArgument, "(argument)", 2, 2)

#Let's initialize TGB (this includes any platform specific stuff, for fonts or whatever)
TGBInitialize()

#We're running the main loop here... neat
while Game.isRunning():
    Game.mainLoop()
    
#Once we're out of the main loop, it is time to shut the system down
Game.mainShutdown()

#Finally, let's clean up any platform specifics
TGBShutdown()

Recent Blog Posts

List:	03/29/08 - TGEA 1.7 Build System and Embedded Python 03/14/08 - MegaTerrains - TGEA Update 01/18/08 - Minions of Mirth: Undead Wars Expansion 01/04/08 - Physics Overhaul - Video 12/26/07 - Web Integration - Video 12/21/07 - New MMO Client - Trees - Day/Night Video 12/18/07 - Minions of Mirth - 1.26 - Holiday Edition! 11/28/07 - TGB/TGEA integration first pass

Submit your own resources!

Jay Barnson (Nov 26, 2007 at 23:16 GMT)

Oh, man. This is pretty sweet...

If I wasn't already halfway through the code of my current project in TorqueScript, I'd be on this in a heartbeat. I've kinda been out of the Python development space for about three years now, and it seems like a lot has changed and improved since then. I'd never heard of SIP before. Is it really as simple as you make it look?

Johnathon (Nov 27, 2007 at 00:20 GMT)

Wow, this is all way over my head but it sounds awesome!

Vashner (Nov 27, 2007 at 01:06 GMT)

Good stuff...

Prairie Games (Nov 27, 2007 at 03:27 GMT)

@Jay: SIP is pretty darned great at generating efficient wrappers. They also compile fast and have a small footprint. The interface language, Python module, and C++ support functions help a lot too for corner cases.

I've written a script which uses GCC-XML (via pygccxml) to automatically generate the interface files. This is using the actual header files from Torque and the project... so if they change, it's trivial to regenerate the interfaces. I've written some C++ code to get things all jiving together. It is getting interesting now.

GCC-XML (and pygccxml which is just a nice way to interact with the XML compiler data) could be used for all kinds of stuff. It is a really cool code analysis system...

Phil Carlisle (Nov 27, 2007 at 09:25 GMT)

Crazy Josh,

How the hell do you find this stuff?

Python still scares me :)

J.C. Smith (Nov 27, 2007 at 13:52 GMT)

Good stuff.

Andy Hawkins (Nov 27, 2007 at 15:01 GMT)

Python is easy!!! V Cool. I wonder though, have you worked out a reload script system so you do basically edit and continue but in Python? I've always found that difficult.

Daniel Staub (Nov 27, 2007 at 19:01 GMT)

I'm lost.

Correct me where I am wrong but this is how I translated your .plan;

GCC-XML -> Custom C++ bit -> pygccxml -> SIP gives Python access to all the TGB and TGEA goodness.

Would this allow you to use 100% Python to write TGB and TGEA based games?

Sammual

bank (Dec 02, 2007 at 14:09 GMT)

Josh, can you tell apprx how fast the Python comparing to the TS?
I'm in situation, when at couple of places (server-side) I need to do something to gain more performance. I'm thinking about moving the stuff into engine/c++, or -- if the python in torque is at least 2-3 times faster than TS, I'll pick it up..
Can you share your experience on that?

ty

Matthias Georgi (Jan 20, 2008 at 12:17 GMT)

This is really interesting!

I developed a complete ruby binding for torque and my approach was to generate the interface from the documentation string, which holds the type information of the parameters (I had to fix some parts of the documentation). The argument types are parsed and each ruby interface method checks the arguments for type safety. Additionally class fields exposed in the engine are accessible by ruby and converted to the right type.

Using this interface I rewrote almost all torque scripts of the tgea distribution and it works quite well. Using your interface generation method, I could generate a tighter ruby binding, which exposes all c++ methods and would circumvent the string conversion of arguments. Are you inclined to share the code?

P.S.: Overriding virtual methods seems to me only achievable by generating a c++ class or am I missing something?

Matthias Georgi (Jan 20, 2008 at 12:20 GMT)

@bank: python should be faster than torque script. I did some silly benchmarks, comparing ruby with torque script and ruby was definitely faster. Python is bytecode-based, so I expect it to be even faster.

You must be a member and be logged in to either append comments or rate this resource.