What is the Script Identifier length limit?

#1 There's no limit defined in the grammar.

In the MKS parser implementation in the TGE SDK, the max size of the token buffer is 4097, so I'd say that the limit is 4096.

If you happen to be generating your own parser with flex... I'm not sure, but at a glance it doesn't look like that implementation uses a separate buffer for the token (rather just using a pointer into the overall scanbuffer), so if that's true there would be no practical limit in this situation.

Either way, looks like you're allowed some mighty big identifiers, unless I've missed something. Doesn't look like the string table has a limitation on entry size other than just needing to have enough space in the memory pool.

#2 Thanks Joel.
I'm currently doing some in-depth in-house documenting of the scripting language and engine functions. I need to do some more looking into the parser and engine code to figure a bunch more of this stuff out...

David

#3 That's great!

Please share with the rest of us when you are done. :-) I assume you've already seen the TGE scripting language reference... I'd appreciate any necessary corrections there.

BTW there are two more related resources in the approval pipeline which may be of some use: list of preferences, list of engine-linked globals

#4 Hi Joel,

Yes, I've basically folded your reference page into what I've got. ;-)
I'm HTML'ing the work, I can upload it to my server and send you the URL if you like.

I'll look at the other two links, thanks.

I'm not sure what will happen to the documentation once I have it completed (besides its in-house use). I always seem to get shot down on everything I try to do or recommend for the community, so I've kept prior work for in-house only. I'll probably email Tim about it anyway though.

David

#5 David,

Jeff T posted on a thread not to long ago reading Script Documentation.

To make a long story short he asked for someone to step and do it ;) well, me being a sucker( or is a fool??) I did ;)

If you wish, you can toss me your info and I will combine it with my documentation (along with the other resources I find), or we can work jointly on it?.

Hopefuly one day we will have a fairly decent community document geared towards the TGE scripting aspect of things.

Regards,
Ron

Here is a link to the post I mentioned above

#6 Hi Ron,

Thanks for the heads-up.

How far along have you gotten, and what area have you started scripting?
Also, what file format are you doing this in?

I started all of this a while ago, documenting some of the class properties etc.
Right now I'm just finishing off a scripting language reference (how to, syntax, keywords, etc.).
Then I'm back on to the classes, functions, global variables, etc.

Feel free to email me privately at dgreen@lilchips.com

David

#7 Hi Joel,

One other thing...
I would assume that their parser builds a function table etc., if the compiled dso files maintain the same names, then using longer names will be slower and require more memory. ??

David

#8 Every time the tokenizer encounters an identifier (which includes function names and variable names), it inserts it into the string table, which is a hashtable. BTW I guess this would be a reason to say that although the scripting can use identifiers instead of strings for a certain restricted format of strings, it's better not to, so that you don't take up space in the string table unnecessarily.

When a function is compiled, a pointer to its name is stored in the bytecode, so its name can be instantly retrieved when the bytecode is executed. So that part doesn't care how long the name is.

However the function name is used when doing a lookup to find which function to execute, since there may be several functions of that name in different namespaces. That lookup speed could vary depending on the name length. But I doubt that it varies enough to be concerned about, if you are using reasonably sized names. If the speed of function lookup for normal function names is taking up any significant time, then I'd say you're probably executing script code WAY too often. :-)

#9 BTW I don't mean to imply that "normal" strings don't take up memory, of course they do... just not (I don't think) in that particular string table that is used for identifiers.

#10 Hi Joel,

Thank you again for your very informative insights, very much appreciated.

David