JIT Coolness
by Frank Carney · 01/14/2012 (12:34 pm) · 9 comments
What if you could get your favorite scripting environment to have JIT? What if there was a high level tool designed to help you make your scripting environment JIT? Guess what, there is one available: PyPy
Some of you are saying: What is JIT?
This is one of the coolest things I have seen in a long time. PyPy is not just a JIT for Python 2.7 code. It is a tool for creating JIT for any scripting language. It uses a subset of Python for defining your scripting interpreter. Once that is written you tell it to produce a version that will JIT. Then the interpreter will perform JIT auto-magically.
I don't know all the details, but it is very interesting. It made me wonder what it would take to build a bytecode interpreter for Torque. Then have it produce a JIT for the bytecode. That could potentially make the Torque Script orders of magnitude faster! You would not necessarily need to jump to C++ to optimize things! It would also be a great thing to put in a product feature list. I don't know if other engines are doing this or not, but it sure would be an attractive feature! This is certainly an exciting time to be a programmer these days!
Holy heck! This is something I was brainstorming and someone already figured it out! This is absolutely amazing! This means I can spawn a process from my main Python code in the PyPy Python JIT. So the spawned interpreter is running like it is compiled code. Now I need to figure out how to talk to the process using C++. Why? Then I can have my Python app that I have Torque running in spawn PyPy processes and have Torque communicate to those processes! Think of it, each AIPlayer could have its "think" function running as a separate process with compiled Python code! So if you have 100 AIPlayers they all think with compiled code! Holy cow this is awesome!
I will figure out a way to have the calls be differentiated as to which part of the call called which. For speed I could just make a function call called extThink as well. That might be better to get rid of the decision making. This way the Torque Script is not waiting for the result so no slowdowns there.
Some of you are saying: What is JIT?
This is one of the coolest things I have seen in a long time. PyPy is not just a JIT for Python 2.7 code. It is a tool for creating JIT for any scripting language. It uses a subset of Python for defining your scripting interpreter. Once that is written you tell it to produce a version that will JIT. Then the interpreter will perform JIT auto-magically.
I don't know all the details, but it is very interesting. It made me wonder what it would take to build a bytecode interpreter for Torque. Then have it produce a JIT for the bytecode. That could potentially make the Torque Script orders of magnitude faster! You would not necessarily need to jump to C++ to optimize things! It would also be a great thing to put in a product feature list. I don't know if other engines are doing this or not, but it sure would be an attractive feature! This is certainly an exciting time to be a programmer these days!
BIG UPDATE:
stackoverflow.com/questions/8659180/multiprocessing-listeners-and-clients-betwee...Holy heck! This is something I was brainstorming and someone already figured it out! This is absolutely amazing! This means I can spawn a process from my main Python code in the PyPy Python JIT. So the spawned interpreter is running like it is compiled code. Now I need to figure out how to talk to the process using C++. Why? Then I can have my Python app that I have Torque running in spawn PyPy processes and have Torque communicate to those processes! Think of it, each AIPlayer could have its "think" function running as a separate process with compiled Python code! So if you have 100 AIPlayers they all think with compiled code! Holy cow this is awesome!
Another Update:
Okay, I can live with not having PyPy talking directly to C++. I will do this:function AIPlayer::think(%this, %data, %objectCalled)
{
// true if called from AIPlayer
if(%objectCalled)
{
feed data to python which feeds PyPy
// data not processed
return false;
}
get data from PyPy process and do normal processing
// data processed
return true;
}I will figure out a way to have the calls be differentiated as to which part of the call called which. For speed I could just make a function call called extThink as well. That might be better to get rid of the decision making. This way the Torque Script is not waiting for the result so no slowdowns there.
Final Edit:
I can use the Python Abstract API to access a Queue object from the multiprocessing library. This is probably as fast as I am going to get barring talking directly to the process using custom C++ code. I could also build some helping TS functions that will allow the think code to talk to the process. I will have to think about the TS interface. Ideally the TS script would control which processes get created and when they get deleted. I could probably inherit from ScriptObject. Wow, from concept to actual solid workable solution. This has been a good day!
About the author
I Started programming in HS and have never stopped. Now an 18 year vet of programming anything from assembler on a NES console to a nuclear waste processing system. If it can be programmed I may have tried to program it!
#2
It looks like someone is working on the JIT part for LUA though:
luajit.org/
So what are the chances to get a JIT for TS bytecode? It would make Torque that much more awesomer.
Once I get Python working well my next target will be Lua if there is enough interest and bribes...
01/14/2012 (10:21 pm)
Maybe, but the reason I am doing this with Python to begin with is there are over 13,000 libraries for Python. Python is the ultimate glue language. The ability to JIT processes in it is a bonus.It looks like someone is working on the JIT part for LUA though:
luajit.org/
So what are the chances to get a JIT for TS bytecode? It would make Torque that much more awesomer.
Once I get Python working well my next target will be Lua if there is enough interest and bribes...
#3
The vast majority of what the interpreter does is calls to engine code, console methods and string manipulation / conversion. JIT would have no effect on those at all. That basically leaves logic and maths. It is true that logic would probably get sped up a bit, but math would be encumbered by still needing lots of string conversions due to the way variables are handled. Some math may get sped up, though.
I do not believe that the difference would be sufficient to be worth the development effort. It would take a similar amount of time and effort to either completely redesign TorqueScript to remove the bottlenecks or replace TorqueScript with an existing language. That would get you far greater performance improvements than implementing a JIT would.
So anyway. Yes, a JIT is possible. No, it is more than likely not worth the effort.
T.
01/16/2012 (2:13 am)
I actually looked into making a JIT for TorqueScript a good 5 or 6 years ago. I grant you that my memory may be hazy on this since it's been a long time since I looked at it, but from what I recall the reasons I didn't bother with it were something along the lines of the following.The vast majority of what the interpreter does is calls to engine code, console methods and string manipulation / conversion. JIT would have no effect on those at all. That basically leaves logic and maths. It is true that logic would probably get sped up a bit, but math would be encumbered by still needing lots of string conversions due to the way variables are handled. Some math may get sped up, though.
I do not believe that the difference would be sufficient to be worth the development effort. It would take a similar amount of time and effort to either completely redesign TorqueScript to remove the bottlenecks or replace TorqueScript with an existing language. That would get you far greater performance improvements than implementing a JIT would.
So anyway. Yes, a JIT is possible. No, it is more than likely not worth the effort.
T.
#4
You are right. TS is very close to the metal. I am not sure it would speed it up replacing the TS with something else either.
01/16/2012 (2:31 am)
@Tom,You are right. TS is very close to the metal. I am not sure it would speed it up replacing the TS with something else either.
#5
The entry point for console functions/methods expect everything as an array of strings. This means it is possible for the interpreter to convert an integer to a string, call a console method, and then said console method converts it right back to an int.
If you use getField()/getWord() to break up a string then the string will get re-scanned for each call.
Anything that expects a string with multiple components (e.g. the vector functions) will call sscanf() on said string to get at the components. This is not in any way fast, and it includes setting a property/field.
Those are just a few examples off the top of my head. There are a load more. It is not possible to fix those without a pretty major redesign of the way Torque works at it's core.
So yeh, tidying up the core would be a massive speed boost if done right. Replacing the script language with something else could also be a big speed boost provided some core refactoring also occurs.
It is worth pointing out, though, that TorqueScript was designed for glue code and some logic with everything performance intensive in C++. When used in that way TorqueScript is plenty good enough. I know you probably know that already, I am just pointing it out in case someone who doesn't comes along later and thinks I'm slating TorqueScript.
This is actually a pretty good example of knowing when and what to optimise and how important profilers are. It is perfectly logical to assume that JIT would speed things up, it's just that in this case the way Torque works internally the bottlenecks are elsewhere.
T.
01/16/2012 (2:49 am)
No, it's not close to the metal. It does a hell of a lot of string manipulation.The entry point for console functions/methods expect everything as an array of strings. This means it is possible for the interpreter to convert an integer to a string, call a console method, and then said console method converts it right back to an int.
If you use getField()/getWord() to break up a string then the string will get re-scanned for each call.
Anything that expects a string with multiple components (e.g. the vector functions) will call sscanf() on said string to get at the components. This is not in any way fast, and it includes setting a property/field.
Those are just a few examples off the top of my head. There are a load more. It is not possible to fix those without a pretty major redesign of the way Torque works at it's core.
So yeh, tidying up the core would be a massive speed boost if done right. Replacing the script language with something else could also be a big speed boost provided some core refactoring also occurs.
It is worth pointing out, though, that TorqueScript was designed for glue code and some logic with everything performance intensive in C++. When used in that way TorqueScript is plenty good enough. I know you probably know that already, I am just pointing it out in case someone who doesn't comes along later and thinks I'm slating TorqueScript.
This is actually a pretty good example of knowing when and what to optimise and how important profilers are. It is perfectly logical to assume that JIT would speed things up, it's just that in this case the way Torque works internally the bottlenecks are elsewhere.
T.
#6
Actually, I already have integrated luaJIT using this resource link. Even though this resource is just for normal Lua, it still works the same way to integrate luaJIT(just include the luaJIT library instead).
I got a HUGE speed increase using the lua script equivalent of the torquescript benchmark code found here. torque script test. LuaJIT in some benchmarks beats java and is approaching the speed of C++ in some cases.
01/16/2012 (12:00 pm)
@Frank Actually, I already have integrated luaJIT using this resource link. Even though this resource is just for normal Lua, it still works the same way to integrate luaJIT(just include the luaJIT library instead).
I got a HUGE speed increase using the lua script equivalent of the torquescript benchmark code found here. torque script test. LuaJIT in some benchmarks beats java and is approaching the speed of C++ in some cases.
#7
No I did not know that. I am not very fluent in the way Torque script works so this is great info. So maybe there is some value in looking at connecting at a C++ level rather than at a console level. Right now I am using functions that look for console objects and methods to call from Python. This sounds like it might be hampered by the console string parsing. Thanks for explaining that. Do you have ideas for working with the engine from another script language? I could do it at the simobject level. Right now I am working from the level to support finding objects and including simobject manipulation. I ask the object if it has methods and call those methods using the execute call.
@Alex,
Cool! That would be a really awesome resource. I keep getting people asking me if I will do Lua next, if it is already done then I don't have to do it! I will remember to point them to Lua instead. Thanks for that.
It is amazing what you learn when you start asking the "right" questions!
01/16/2012 (11:03 pm)
@Tom,No I did not know that. I am not very fluent in the way Torque script works so this is great info. So maybe there is some value in looking at connecting at a C++ level rather than at a console level. Right now I am using functions that look for console objects and methods to call from Python. This sounds like it might be hampered by the console string parsing. Thanks for explaining that. Do you have ideas for working with the engine from another script language? I could do it at the simobject level. Right now I am working from the level to support finding objects and including simobject manipulation. I ask the object if it has methods and call those methods using the execute call.
@Alex,
Cool! That would be a really awesome resource. I keep getting people asking me if I will do Lua next, if it is already done then I don't have to do it! I will remember to point them to Lua instead. Thanks for that.
It is amazing what you learn when you start asking the "right" questions!
#8
I am not really all that familiar with Python so I can't really tell you the best way to integrate it. However, here's some generalised stuff that may or may not be of any help.
Python was designed as a system scripting language for hacking up tools etc, much like Perl. It was not designed to be embedded in other things, rather it was designed to be extended by other things. The difference is subtle, but important.
I believe that past integrations of Python and Torque have been implemented by way of using Torque as a library in Python, but I may be wrong about that.
Languages like Lua or Squirrel are designed to extend other software, rather than as standalone languages. This means they are really, really good at being embedded but kinda suck as standalone languages.
The best way to integrate a different scripting language depends alot on the language and what you want to achieve. It seems to me that the most useful way would be to come up with an alternative for the ConsoleMethod macros so that Torque objects can be used in the other scripting language. With care it is possible to write the templates/macros such that both the old and new can be used.
I am not really sure if that helps you any, though. This is a big and complicated subject :) I would suggest you spend some time learning how the console works internally, as that's something you really need to understand if you want to replace it with something else. Literally everything depends on it and is intricately linked to it.
T.
01/16/2012 (11:56 pm)
Frank,I am not really all that familiar with Python so I can't really tell you the best way to integrate it. However, here's some generalised stuff that may or may not be of any help.
Python was designed as a system scripting language for hacking up tools etc, much like Perl. It was not designed to be embedded in other things, rather it was designed to be extended by other things. The difference is subtle, but important.
I believe that past integrations of Python and Torque have been implemented by way of using Torque as a library in Python, but I may be wrong about that.
Languages like Lua or Squirrel are designed to extend other software, rather than as standalone languages. This means they are really, really good at being embedded but kinda suck as standalone languages.
The best way to integrate a different scripting language depends alot on the language and what you want to achieve. It seems to me that the most useful way would be to come up with an alternative for the ConsoleMethod macros so that Torque objects can be used in the other scripting language. With care it is possible to write the templates/macros such that both the old and new can be used.
I am not really sure if that helps you any, though. This is a big and complicated subject :) I would suggest you spend some time learning how the console works internally, as that's something you really need to understand if you want to replace it with something else. Literally everything depends on it and is intricately linked to it.
T.
#9
Thanks Tom. For T3D I upgraded the original work done by Prairie Games here: www.garagegames.com/community/resources/view/21370 This is a Python extension. It appears as a module to Python.
I am working on my own extension of Python using SWIG. It is not as close to the metal as the original, but I am learning a lot. It was never intended to boost the speed of a scripting language. It is to extend the libraries available. I do a lot of Python contract work so learning how to extend Python is a good thing. Python can be embedded, but that is not what I was looking to do.
I am starting to understand some of it, enough to add a new callback type so I can get namespace info in my Python callbacks. One of my blogs covers that. Just search for SWIG and you should find them.
Mainly I want to extend the libraries available to Torque. I am also fascinated with multiprocessing in general. So getting the chance to use it to speed things up is just awesome. My intent never started with JIT, I just have to share this cool stuff I am learning.
When I get my code up to a certain point I will be releasing a new Python resource for T3D. At that point I will want to get a lot more input on what could be improved. So, I probably should just get my butt in gear and get something to everyone! Haha!
01/17/2012 (12:54 am)
@Tom,Thanks Tom. For T3D I upgraded the original work done by Prairie Games here: www.garagegames.com/community/resources/view/21370 This is a Python extension. It appears as a module to Python.
I am working on my own extension of Python using SWIG. It is not as close to the metal as the original, but I am learning a lot. It was never intended to boost the speed of a scripting language. It is to extend the libraries available. I do a lot of Python contract work so learning how to extend Python is a good thing. Python can be embedded, but that is not what I was looking to do.
Quote:This is what I was asking about. So this does help. What I am having struggles with is what functions to support. I do think it makes sense to have the concept of manipulating the console directly in Python. However I want the objects to be extensible for the SimObjects and down the inheritance chain, but I am having difficulty seeing how I can do that without the dynamic nature of the console. Right now I am using methods for the SimObject that check for Attributes and Functions existence before I call attempt to call them. Otherwise I throw errors back to the Python code. However, ultimately these calls use the Con::execute methods that rely on the determination of what function call to make using string methods.
It seems to me that the most useful way would be to come up with an alternative for the ConsoleMethod macros so that Torque objects can be used in the other scripting language. With care it is possible to write the templates/macros such that both the old and new can be used.
I am starting to understand some of it, enough to add a new callback type so I can get namespace info in my Python callbacks. One of my blogs covers that. Just search for SWIG and you should find them.
Mainly I want to extend the libraries available to Torque. I am also fascinated with multiprocessing in general. So getting the chance to use it to speed things up is just awesome. My intent never started with JIT, I just have to share this cool stuff I am learning.
When I get my code up to a certain point I will be releasing a new Python resource for T3D. At that point I will want to get a lot more input on what could be improved. So, I probably should just get my butt in gear and get something to everyone! Haha!
Employee Richard Ranft
Vorpal Bunnies