TSE, C#, .NET Framework, and Python
by Prairie Games · 04/10/2006 (12:55 pm) · 16 comments
The Torque Shader Engine has a script interface that is built on strings. This makes it quite easy to extend. In the course of a few days, I have written a bridge from TSE to Microsoft's CLR via managed C++. This makes it possible to write our logic using C# and the .NET framework. We can also use Python via Microsoft's IronPython (or under Python for .NET), with no additional code or bindings. The bridge is also quite fast...
C# and Python are highly productive languages and complement each other very well. C# can be used where execution speed is critical. Python can be used where it's not so important and programming time is at a premium. Furthermore, we now have access to the full .NET framework for game and business logic.
The first thing was to break out the engine's mainloop and compile TSE as a mixed mode assembly:
C#
Python
TorqueScript callbacks are defined in much the same way as in C++ and can use namespaces
C#
Python
Now, we can setup a button in the Torque gui builder or via script and set it's command field:
We can also retrive objects, set fields, and execute methods from Python and C#:
C#
Python
C# and Python are highly productive languages and complement each other very well. C# can be used where execution speed is critical. Python can be used where it's not so important and programming time is at a premium. Furthermore, we now have access to the full .NET framework for game and business logic.
The first thing was to break out the engine's mainloop and compile TSE as a mixed mode assembly:
C#
using TSE.Native;
namespace TSE.Managed
{
class Program
{
static void Main(string[] args)
{
Runtime.Initialize();
while (Runtime.Tick() != 0) {}
Runtime.Shutdown();
}
}
}Python
import clr
clr.AddReference("TSE")
from TSE.Native import Runtime
def main():
Runtime.Initialize()
while Runtime.Tick(): pass
Runtime.Shutdown()TorqueScript callbacks are defined in much the same way as in C++ and can use namespaces
C#
static void OnButton(int argc, String[] argv)
{
msg = argv[1];
Console.Write(msg);
}
Runtime.ExportFunction("CSharp::OnButton", 2, 2, new Runtime.TSECallback(OnButton));Python
def OnButton(argc,argv):
msg = argv[1]
print msg
Runtime.ExportFunction("Py::OnButton",2,2,OnButton)Now, we can setup a button in the Torque gui builder or via script and set it's command field:
MyButton.command = 'CSharp::OnButton("Hello from C#!");' <-- call into our C# callback
MyButton.command = 'Py::OnButton("Hello from Python!");' <-- or call into our Python callbackWe can also retrive objects, set fields, and execute methods from Python and C#:
C#
//this is purely an example
TSEObject MyButton = TSEObject.findObject("MyButton");
MyButton.setField("command","CSharp::OnButton(\"Hello from C#!\");");
MyButton.execute("performClick");Python
#python's dynamic nature makes it very nice to work with...
MyButton = TSEObject.findObject("MyButton")
MyButton.command = 'Py::OnButton("Hello from Python!");'
MyButton.performClick()
#2
04/10/2006 (2:11 pm)
I'd be interested in seeing the bridge code too. I've been planning on doing this myself for some time now, just waiting on available time to work on it. In fact I downloaded your TGEPython resource just today for reference.
#3
04/10/2006 (2:55 pm)
This, Sir, sounds very cool! :) Looking forward to a resource, Josh! 
#4
04/10/2006 (3:50 pm)
Josh, you make me jeaulous ;)
#5
what josh is describing is a method of interop, similar to COM Interop.
The reason I'm saying that, is I dont want people to confuse this with my torque.net work. My stuff is a direct replacement for torquescript (and allows interop almost identical to what Josh described)
If you want to do this interop stuff Josh is mentioning, it's very easy to do, if you are already a c++ and C# programmer. Just look up PInvoke for how to talk from C# to C++, and look up C++/CLI for how to talk from C++ to C#.
Okay.. well Ive already mentioned the above stuff a bunch of times in other threads, and I know If I talk more about the topic I'll probably get defensive (as t2d.net is my baby) so I'm going to refrain from beating the horse and I wont be replying to these topics again... at least not until I get done with my t2d.net CTP2 stuff in the middle of May.
-Jason
04/10/2006 (5:54 pm)
In case there is any confusion, what josh is describing is a method of interop, similar to COM Interop.
The reason I'm saying that, is I dont want people to confuse this with my torque.net work. My stuff is a direct replacement for torquescript (and allows interop almost identical to what Josh described)
If you want to do this interop stuff Josh is mentioning, it's very easy to do, if you are already a c++ and C# programmer. Just look up PInvoke for how to talk from C# to C++, and look up C++/CLI for how to talk from C++ to C#.
Okay.. well Ive already mentioned the above stuff a bunch of times in other threads, and I know If I talk more about the topic I'll probably get defensive (as t2d.net is my baby) so I'm going to refrain from beating the horse and I wont be replying to these topics again... at least not until I get done with my t2d.net CTP2 stuff in the middle of May.
-Jason
#6
Definately interested to see where this goes
04/10/2006 (6:28 pm)
Very interesting.. another Engine that I am licensed for uses C# as its Scripting interface completely. Although it does its own compiling and doesn't require anything extra which is nice..Definately interested to see where this goes
#7
04/10/2006 (9:12 pm)
@Josh - This is pretty neat, but what benefits do you see from C#/Python over TorqueScript? Is it tools? Performance? Familiarity and available documentation? Or is it just OOP language features?
#9
I'm just wondering what sort of cross platform ability C# has to run, because it would be nice to bind tools for comms into a C# backend (i.e. db tools) and access those from TSE.
Cool stuff though.
04/11/2006 (1:07 am)
I actually really like C# as a language. To be able to run TSE under C# control is pretty nice. But does this mean that TSE is basically a C# object?I'm just wondering what sort of cross platform ability C# has to run, because it would be nice to bind tools for comms into a C# backend (i.e. db tools) and access those from TSE.
Cool stuff though.
#10
On the other side of the fence, none of the windows specific stuff works in Mono. For example the Windows Forms libraries for Mono don't work great at all, and they will most likely never work well. To make things even more confusing, every time Microsoft creates a new version of .NET, they rewrite about half the platform for no good reason. So Mono is always lagging behind Microsoft's implementation of new API's quite a bit. The core Mono platform is pretty solid, but slower than Microsoft's implementation of .NET.
So to be honest, if you were going to use .NET, you would most likely want to embed the whole Mono platform in your application, rather than using the host's version of .NET. This would solve most of the biggest deployment / cross platform issues.
But then we have yet another problem which is specific to a Torque / Mono integration. Mono doesn't have Managed C++, which is the only langauge that can be used to embed C++ code into mono. So essentially, you can't create an integrated Torque / Mono platform. Well I am lying a little bit. You could create a Torque / Mono platform, but that would require wrapping every single API in Torque, into a standard C based object system.
Well anyways its an interesting idea, but I don't think its a very good idea.
04/11/2006 (8:58 am)
Mono works pretty good on Linux and Mac OS. If you want to check it out, you can download it at [link]http://www.go-mono.org[/link]. C# is an interesting language, but it has a lot of issues. Every time Microsoft updates the platform, they increase the version number, and your program will only run under the version of .NET that its compiled for. This is very confusing for end users, since they have to have the exact version of .NET that your application was compiled under, in order to run it (there are three versions of .NET already).On the other side of the fence, none of the windows specific stuff works in Mono. For example the Windows Forms libraries for Mono don't work great at all, and they will most likely never work well. To make things even more confusing, every time Microsoft creates a new version of .NET, they rewrite about half the platform for no good reason. So Mono is always lagging behind Microsoft's implementation of new API's quite a bit. The core Mono platform is pretty solid, but slower than Microsoft's implementation of .NET.
So to be honest, if you were going to use .NET, you would most likely want to embed the whole Mono platform in your application, rather than using the host's version of .NET. This would solve most of the biggest deployment / cross platform issues.
But then we have yet another problem which is specific to a Torque / Mono integration. Mono doesn't have Managed C++, which is the only langauge that can be used to embed C++ code into mono. So essentially, you can't create an integrated Torque / Mono platform. Well I am lying a little bit. You could create a Torque / Mono platform, but that would require wrapping every single API in Torque, into a standard C based object system.
Well anyways its an interesting idea, but I don't think its a very good idea.
#11
You are absolutely right about the Windows specific libraries not being implemented. I did see an article saying that the Mono team is now working on a complete Forms implementation, but who can say how it will turn out.
I don't really agree that they "rewrite half the platform for no good reason". I've just recently ported a commercial application that is in development from 1.1 to 2.0 and while there is many changes, the existing code, in most cases, works unchanged. Even the volume of deprecation warnings is relatively low. And I personally find Generics and partial classes (2 of the new features in 2.0) very useful.
The argument about embedding C++ code into Mono seems like a backwards approach to me. I will be embedding a CLI runtime into Torque, rather than the other way around. See here for the relevant API's.
04/11/2006 (9:13 am)
I'll have to disagree with most of your points Ray. While it's true that compiled assemblies target a particular version of the framework, it is possible to create assemblies that will run under any/all versions, or a particular minimum version, such as 1.0 or 1.1 etc. The applicable information is hereYou are absolutely right about the Windows specific libraries not being implemented. I did see an article saying that the Mono team is now working on a complete Forms implementation, but who can say how it will turn out.
I don't really agree that they "rewrite half the platform for no good reason". I've just recently ported a commercial application that is in development from 1.1 to 2.0 and while there is many changes, the existing code, in most cases, works unchanged. Even the volume of deprecation warnings is relatively low. And I personally find Generics and partial classes (2 of the new features in 2.0) very useful.
The argument about embedding C++ code into Mono seems like a backwards approach to me. I will be embedding a CLI runtime into Torque, rather than the other way around. See here for the relevant API's.
#12
"embedding C++ code into Mono seems like a backwards approach to me"
In my universe, TSE is a component to be combined with technology that I write... I do not write game and business logic in C++
04/11/2006 (10:41 am)
I am being careful to use stuff that will run under Mono... It is completely possible to generate a syntax compatible bridge, without using managed C++, once the need is there. "embedding C++ code into Mono seems like a backwards approach to me"
In my universe, TSE is a component to be combined with technology that I write... I do not write game and business logic in C++
#13
That was for Ray's benefit, and you are right, the whole draw of implementing a CLI runtime is use it for business and game logic. Don't get me wrong, I have a lot of respect for the work you've done, just offering a different possible approach.
I was referring to this:
"Mono doesn't have Managed C++, which is the only langauge that can be used to embed C++ code into mono."
Which sounds backwards to me, granted Mono doesn't offer managed C++, but the same could be accomplished by implementing the bridge in C# compiled under Mono and using P/Invoke. The backwards part is in the assumption that the Bridge must be compiled into the same library as TSE.
If/when I ever get time to work on it, the only reason I chose to approach it by creating the AppDomain(s) from within TSE using the C++ api into Mono/.NET is that I can more closely model how the engine interfaces to torquescript.
04/11/2006 (10:54 am)
@JoshThat was for Ray's benefit, and you are right, the whole draw of implementing a CLI runtime is use it for business and game logic. Don't get me wrong, I have a lot of respect for the work you've done, just offering a different possible approach.
I was referring to this:
"Mono doesn't have Managed C++, which is the only langauge that can be used to embed C++ code into mono."
Which sounds backwards to me, granted Mono doesn't offer managed C++, but the same could be accomplished by implementing the bridge in C# compiled under Mono and using P/Invoke. The backwards part is in the assumption that the Bridge must be compiled into the same library as TSE.
If/when I ever get time to work on it, the only reason I chose to approach it by creating the AppDomain(s) from within TSE using the C++ api into Mono/.NET is that I can more closely model how the engine interfaces to torquescript.
#14
Do you know?
04/12/2006 (1:00 pm)
@Matthew: you have a nice point about embedding CLI into an application. What are the legal problems that can appear, linking mono as a lib? I tried to find it (license), but couldn't...Do you know?
#15
I'm fairly certain it's free from legal restriction since the embedding happens via the use of the header files only, and the link can be dynamic rather than static. Two important distinctions when qualifying for use of the LGPL.
I'm a big fan of abstract interfaces and modular/pluggable designs that support extension and replacement of technology in a clean and easy fashion. I think that same design can be applied here to reinforce the use of the LGPL, since one of the qualifiers is usually that the product that links the covered libraries be able to function without those libraries equally well. So in this case having an abstract interface that would allow the engine to load both Mono and .NET runtimes would satisfy this requirement.
All that being said, I generally try to err on the side of caution when dealing with licenses, as the last thing I want some day down the road is to have my hard work and IP tarnished by licensing violations during its development. When I get this working to my satisfaction, I'll probably contact the Mono developers and ask them to review the integration details and get a licensing ruling to be sure the work falls under acceptable use.
04/12/2006 (1:15 pm)
It's hard to say really. I'm certainly not a contract lawyer, and as far as legal documents go, I've never found the GPL/LGPL to be a shining example of readability.I'm fairly certain it's free from legal restriction since the embedding happens via the use of the header files only, and the link can be dynamic rather than static. Two important distinctions when qualifying for use of the LGPL.
I'm a big fan of abstract interfaces and modular/pluggable designs that support extension and replacement of technology in a clean and easy fashion. I think that same design can be applied here to reinforce the use of the LGPL, since one of the qualifiers is usually that the product that links the covered libraries be able to function without those libraries equally well. So in this case having an abstract interface that would allow the engine to load both Mono and .NET runtimes would satisfy this requirement.
All that being said, I generally try to err on the side of caution when dealing with licenses, as the last thing I want some day down the road is to have my hard work and IP tarnished by licensing violations during its development. When I get this working to my satisfaction, I'll probably contact the Mono developers and ask them to review the integration details and get a licensing ruling to be sure the work falls under acceptable use.
#16
04/14/2006 (11:59 am)
That's a good answer, Matthew, I'll try to contact Mono team and ask them this question. 
Torque Owner David Miller
Default Studio Name
I have wanted to try the same thing but just really didnt know where to start on the TSE side.
Dave