Python: T3D Console Data Model

This is part of my series of coding blogs surrounding my trek through learning how to extend Python with T3D using SWIG. My goal is to present interesting coding subjects as I make my way through the process.

Today's edition deals with how to abstract console objects, variables, and methods. I spent a lot of time trying to figure out how to extend the attribute methods of a Python object, only to find a much cleaner and workable solution by using container methods.

What was I trying to solve?
I wanted to be able to access console methods, variables, and objects with functions like this:

static SimObject * FindObject(S32 param);
static SimObject * FindObject(const char *param);
static const char* GetVariable(const char *name);
static bool SetVariable(const char* name, const char* value);
static bool IsFunction(const char *nameSpace, const char *name);
static const char* Evaluate(const char* code);

Those functions still need to exist, but I wanted to do this:

pyT3D.Sim().function()
val = pyT3D.Sim().globalvar
pyT3D.Sim().globalvar = 5

I wanted a more natural interface to the objects. So I set out using __getattribute__ and __setattr__ methods. Those methods are designed to allow access object using dot (.) operators. However, after actually implementing those and figuring out how to actually make those work I found my concept failed with __setattr__. The reason is because this is mechanism used to set variables for the object itself. If I give precedence to the objects default methods it will set the values for the object, if I give precedence to the console methods then it will set the console variables. The problem is the Python object must be able to set attributes on itself to work properly. Python objects are highly dynamic which is very similar to the dynamic nature of console objects. So the two methods are mutually exclusive, damn it!

At that point I pouted, swore, kicked some things. I had spent hours trying to understand how to use the
__getattribute__ and __setattr__ methods properly. My getattribute function worked perfectly, but conceptually the method was to fail. Then I started looking at all the different methods built into python objects here. That is when I found __setitem__ and __getitem__. These would allow me to use my object like a container that uses strings for lookups! What a perfect match! So in about 10 minutes I had tweaked the code, removed all the junk needed to make the other methods work, and I had working abstraction for my console objects for both set and get methods:

pyT3D.Sim()["function"]()
val = pyT3D.Sim()["globalvar"]
pyT3D.Sim()["globalvar"] = 5

Why are these methods better? Because I can use any string that is a valid console syntax. With the dot (.) operators I could not do that. Python has certain characters that cannot be used for variable names such as "::","$", and "%". The console relies on those. With the container methods I can do this:

pyT3D.Sim()["$fps::missiontime"] = "10:00"

I could not do that with the dot (.) operators.

So to give you an idea of what part of the swig code looks like:

%pythoncode %{
def BuildExecString(objName,name,*args):
	targs = ""
	amax = len(args)
	for i in range(amax):
		arg = args[i]
		targs += '"'+str(arg)+'"'
		if(i < amax-1):
			targs += ","
	if objName is not None:
		fcall = "return "+objName+"."+name+"("+targs+");"
	else:
		fcall = "return "+name+"("+targs+");"
		
	return fcall
%}

%rename("Sim") "SimObjs";
class SimObjs {
public:
	static SimObject * FindObject(S32 param);
	static SimObject * FindObject(const char *param);
	static const char* GetVariable(const char *name);
	static bool SetVariable(const char* name, const char* value);
	static bool IsFunction(const char *nameSpace, const char *name);
	static const char* Evaluate(const char* code);
	
	%pythoncode %{
	def __setitem__(self, key, value):		
		setattr = Sim.SetVariable(key, value)
		if setattr:
			return
			
		raise AttributeError(key,"is not a valid console variable.")
	
	def __getitem__(self, key):
		key = str(key)
		
		# find attribute in console if possible
		sobj = Sim.FindObject(key)
		if sobj is not None:
			return sobj
		svar = Sim.GetVariable(key)
		if len(svar):
			return svar
		
		if "::" in key:
			fname = key.split("::")
		elif "." in key:
			fname = key.split(".")
		else:
			fname = [key]
		if len(fname) == 1:
			fbool = Sim.IsFunction(None,fname[0])
			if fbool:
				def tfunc(*args):
					#print "function call:",fname[0]
					fcall = BuildExecString(None,fname[0],*args)
					ret = Sim.Evaluate(fcall)
					return ret	
				return tfunc
		elif len(fname) == 2:
			fbool = Sim.IsFunction(fname[0],fname[1])
			if fbool:
				def tfunc(*args):
					#print "function call:",fname[0]+"."+fname[1]
					fcall = BuildExecString(fname[0],fname[1],*args)
					ret = Sim.Evaluate(fcall)
					return ret	
				return tfunc
				
		# if an attribute cannot be found
		raise AttributeError(key,"is not a valid console variable.")

	def __str__(self):
		return "Sim interface for console methods."
	%}	
};

At the beginning of this code is a Python function called BuildExecString. This helps me build a wrapper function for console functions. The idea is I can feed it a function name and an arg list and it will build a string to be exec'd by the Con::evaluate function. This has been the most reliable way that I can find to call console functions.

The next part is where I define the Sim class. I had to rename another class to keep it from colliding with the Sim namespace in Torque itself. Inside the Sim class I have some static methods to do the various interfacing to the console. They are pretty self descriptive.

Then I define the __setitem__ and __getitem__ methods using Python code. It is more efficient coding wise and brain cell wise to implement these methods in Python. The setitem method uses SetVariable at it's core. This function only affects variables, not objects or functions. The SetVariable function actually returns a boolean which is True if the variable was actually set. The getitem method first searches for a SimObject, then a variable, and finally a function. It will return the objects/values if found.

What is interesting about the function return is there is really no object or value to return, but it must return a callable Python object or Python function to be able to executed. So I create a Python function on the fly, use the BuildExecString to build an executable console string, embed that data in the function, and return a reference to that newly created function. When the function is called it will use the Evaluate method to execute the function with its parameters already built into the string. That is one thing I love about Python. You can just create functions out of thin air! This is great for abstracting data interfaces without a Python compatible object model.

Well that is it for this segment of brain damage. Stay tuned for more exciting data abstraction!