TGE: Twisted Python distributed/internet framework

#1 Nice! I was going to have to implement a lot of the stuff in twisted matrix for my project... *reads further of twisted* Hot damn. You just saved me months of dev time. :)

I don't suppose you're planning on releasing that reactor, hmm? ;)

(I hope everyone who asked, why add Python, sees this. It'd be... hmm... tricky to do all this in TorqueScript ;)

#2 Do you think Twisted would be fast and scaleable enough for a Torque based MMPOG, driven by a back end database?

This does sound extremely interesting.

#3 Resource?

#4 I agree with Edward. Submit this as a resource.

#5 Some things will be proprietary to the (unannounced) project I am working on... but I will make an effort to keep code seperation for the stuff I want to release...

Below is pasted the (trivial) twisted reactor, which is a modified version of their gtk reactor... it simply hooks Twisted into the Torque main loop via schedule... notice that it is set to only run at 10 times a second... for my Login service this is more than adequate... for other uses you may have to lower granularity...

I have a client with Python (and not .cs) in charge... I simply PyExec() some python that installs this as per Twisted documentation... Twisted will work equally as well with .cs script... just PyExec() from the .cs in this case :)

If you do play with this... make sure that the FPU is in double precision when calling into Python in TGEPython.cc ... under windows this is:

_controlfp( _CW_DEFAULT, 0xfffff );

There are some things I have added to TGEPython ... like ./ notation from python which conforms to TGE's mod paths... I got rid of a debugging crumb which only allowed GameBase objects into Python callbacks(noticeable when coding Torque GUI's in Python *grin*)... added a flush to the stdout writer(twisted likes this)... etc... I'll see about packaging this up soon...

One other note... I have been messing around with SQLite and PySQLite today... seems to play along nicely with the twisted enterprise database stuff... I'd rather use something like this than MySQL or PostgreSQL me thinks :)

pysqlite.sourceforge.net
www.hwaci.com/sw/sqlite

-J

-----TorqueReactor.py----

from tgenative import *
from tgepython.console.simobject import TGEExport

__all__ = ['install']

import sys

import twisted.internet.default as default

gReactorTickId=None
gReactor = None

def ReactorTick():
    global gReactor
    global gReactorTickId
    gReactor.simulate()
    timeout = min(gReactor.timeout(), 0.1)
    if timeout is None:
        timeout = 0.1

    gReactorTickId=int(TGECall("schedule",1000*timeout,0,"ReactorTick"))
    

class TorqueReactor(default.SelectReactor):

    def crash(self):
        pass
        
    def run(self):
        self.startRunning()
        ReactorTick()
        
        
    def simulate(self):
        """Run simulation loops and reschedule callbacks.
        """
        self.iterate()

def TGEInstall():
    """Configure the twisted mainloop to be run inside the Torque mainloop.
    """
    
    global gReactor
 
    TGEExport(ReactorTick,None,"ReactorTick","ReactorTick()",1,1)
    
    reactor = TorqueReactor()
    gReactor=reactor
   
    from twisted.internet.main import installReactor
    installReactor(reactor)

    return reactor

install = TGEInstall

#6 Darn spiffy :)

#7 I'm using Twisted (and have been for almost a year) for a scalable MMP backend. I'll be talking a bit about it during the breakout session I'm hosting at IndieGamesCon, Sat at 2:45pm.

It's been a godsend, let me tell you :)

#8 Yuppers... and now it's 1.0 :)

www.zoteca.com/Press/prtwisted10.html

-J

#9 Jeremy, will there be anything we can read after this talk?

#10 I'll certainly post notes/slides/etc, and I think someone is going to be taping it. I'm of course always happy to take emailed questions about specifics though.

#11 Greetings!

Thanks for posting this information and letting us all know about Twisted. It certainly looks like a great library and I'm certain it will save me a lot of coding time.

Jeremy, as I will not be attending IGC, I certainly would be interested in a copy of your materials. Thanks!

- LightWave Dave

#12 That's a session i plan on attending :)

#13 Check out passing deferreds OVER THE WIRE... ie. you can return them from a remote function call... the docs don't hammer in how insanely cool this is...

I was using pb.Referenceable instances as arguments to remote functions and then remote invoking back on these from a local deferred... I didn't realize I could just pass a deferred back, from the original remote function...

BRAIN MELTS... OOZES EVERYWHERE...

I have a client -> master <-> world <-> zone architecture with a Firebird (Interbase) database backend...

Remote function calls, requests to the database, TGE UDP networking, all flying around and coming back when they are done... insanity...

-J

#14 I'm not sure if you were suffering brain overload there Joshua, but that's exactly what I want. I mean, the client-master-zone-architecture, not the brain overload!

#15 ---

#16 Here's a nice start:

diveintopython.org

Now back to day and night cycles... feels good to be doing some OpenGL for a change of pace :)

-J