Game Development Community

TGE 2.0 feature wishlist

by D Player · in Torque Game Engine · 03/07/2003 (2:48 am) · 69 replies

Alright, this thread is only about what changes/features you want to see in TGE 2.0, and the reasons. Now, other people seem to believe that the GG folk will descent from on high and tell us how things will be. Even if they have the time and inclination, this will turn out better if we discuss things. Let's find out what's good and what's bad. I'll start this thread off with features that I want to see.

1. A rearchitecting of TGE. I think we can all admit that the current engine is not well broken up into replaceable components, largely because many components are tightly coupled (either through #includes, or misplaced functionality). The real problem here is that to make any changes you have to read, understand, and take into account FAR too much code that will be inadvertently effected.
2. Reduce code duplication. There are plenty of places in the engine where the same thing is done in different places, ever so slightly differently. This could probably be simplified.
3. Assert. I've seen very little of this in use, Assert is an essential tool for catching bugs and logic errors. These need to be spread liberally throughout the engine. Also, these largely stand in for documentation in some cases. They stay both current and right in the code, while documenting the expected (pre)conditions.
4. Curved surfaces integrated tightly into a LOD system. Without curved surfaces, an artist created LOD system compensates for this lack at the cost of memory, effort, and visual quality.
5. Official replacement of cs with Python, or a performance analysis that shows this as being an unreasonable course of action.
6. Shaders; just because people will complain about this endlessly unless they are on the plan. At least the architecture to support shaders easily must be done.
7. More generic engine. TGE is really, truly a FPS engine. I realize that making something too generic will make it useless, but there is a lot of middle ground between here and there.


Also, what support structure do we need in place to get things happening?
1. Leadership that makes the final call on features and goals. This person needs to be cocky; don't allow decisions to be made by the community unless there is broad consensus. Just because one group browbeat another group into silence doesn't make them right. This person needs to understand the issues and make the calls. Good communication is a big plus.
2. A roadmap; a plan.
3. A group of core developers, who work on the most critical items
4. Checkin reviewers.

Seriously here, let's get this started. There is no reason to wait for others to do things that they may or may not get around to. If GG comes in and helps steer the decision making along with their insight, so much the better, but let's not depend on it.
#41
03/08/2003 (4:17 am)
I have to ask... Are you coming up with these rediculous rebuttals for enteratinment value alone or what?


Quote:As such, you can come right here and learn from the tutorials and various posts on how to script in the Torque Scripting Language. Furthormore, Python books bought in stores won't teach you how to rewrite the scripts currently in place in the engine, nor will it teach you Torque specific scripting related problems and solutions. Since Torque Scripting is such an intrigal part of the engine, when you learn how to script in TGE Script you also learn how to be productive in TGE right away, whereas in Python you would have to learn the language first, and then learn Torque specific solutions.

Trying to compare the handful of tutorials here with PUBLISHED books and THOUSANDS of websites dedicated ot the language is rather silly don't you think? And the individual developer will never NEED to learn how to convert most of their scripts as folks like Josh and myself who are working with things currently will be doing just that before it's sprung on people.

And your last point is so contrived I had to read it 3 times before I got what you were trying to say. TGE script is a LANGUAGE once you learn it you have to learn how to use the TGE objects. Python is a LANGUAGE once you learn it you have to learn how to use the TGE object. The differences are 1) The Python interfaces can be much cleaner and clearer. 2) You can use the knowledge / language for something other than games using TGE.


Quote:The same applies to TGE Script. People submit resources everyday on how to do a specific thing in the TGE Script. And since these resources are immediatly applicable to the engine, there is no need to 'make it work' with the engine, unlike Python where these public class libraries may need adoption in order to work with the engine correctly.

Again it's a quesiton of scale. 100s over 1,000s. And again you're contriving things. It the TGE language was Python the script samles on the site would be in Python. You act like the community is going to disappear and GG would supply ZERO scripts.


Quote:
How is this any different with our 'propritary' TGE Script? I seem to be able to move my script game logic from windows to linux to mac just fine, try it, you'll see.

Funny, my design for GORPE allows me to take 80% of the game and 100% of the game LOGIC to an entirely different engine but then again you skipped the part of the sentence where I said PLATFORM.


Quote:
I don't agree. TGE Script is a compiled language. When it is compiled it is enhanced to maintain the maximum in performance and the maximum in security, because it is built specifically for the engine, and as such, provides the cleanist backend you can get from script to code.

Three letters for you "pyc". TGE script is compiled to bytcode which gets interpreted, so does Python except Python's compiler is far more mature and the language is more complete.

As for the cleanist backend to the engine. Have you actualy looked at what's reuired to expose and use C++ objects in TGE script as opposed to Python? Here's an exercise: Add a matrix vairable to the AIPlayer base class, let's call it "foobar" now expose this variable to both scripting languages. Now compare the amount of code you wrote to expose a new variable. Next: Now go into the respective scripting languages and see how much code you write to manipulate that variable.


Quote:I don't agree once again. The game industry seems to be moving more to opening up their engines to modification, yes, but to open languages that aren't propritary, no. The latest example is Unreal Tournament 2K3, its scripting language is propritary, resembling Java. The only difference between it and TGE Script is TGE Script more closely resembles C/C++(a language more game programmers are already familiary with).

And yet everyday we here of this engine or that engine switching to open languages. But this whle discussion is getting REALLY old. I cannot fathom why you insist on taking a stand against an establish general purpose language over a propritary pseudolanguage but frankly I don't care.
#42
03/08/2003 (4:38 am)
Comparing TorqueScript and Python at a language level is futile... if you know even a little about each you'll realize this...

On the backend... I have quite a bit of experience with the internals of this... on both sides of the fence...

Torque's console system is the way you expose C++ code to TorqueScript... bound functions pass strings back and forth... you can bind variables and functions... you CANNOT bind classes (you create instances)... there is no concept of a TorqueScript class... or inheritance... etc... most of TorqueScripts functionality relies on C++ stub functions which call into engine C++ code... I am sensing a reoccuring theme here... C++, C++, and C++

Python has numerous ways to bind C++ code: the Python binding API, Boost.Python templated binding, automated binding via Swig, etc .. bound C++ classes are possible and you can inherit from them in Python... you can even override C++ member functions with Python ones on a per instance basis!!!, use a (vast) standard library, hook up with some other technology using Python... stick it in your sock... light your head on fire and run into the street.. it goes on and on...

As far as language complexity... Python goes from the very simple to advanced flow control/logic, metaclasses, and other exotica... thing is, you can grow with it... or not, it's at least an option... regardless, Python is strong enough that you only need to dip into C++ when speed/memory is an issue... and in this case you can probably find a nice, well documented, extension module some nice group of coders has made and Open Sourced for you...

Anyway, chances are .cs will remain the official scripting language of Torque... which doesn't overly pain me... I'll just use Python :)

-J
#43
03/08/2003 (6:45 am)
You guys are spinning your wheels on the script thing, and should move it to another thread, the original poster wants to hear your "wishes" of what you want to see go into torque, not long debates of what script is better.
#44
03/08/2003 (8:10 am)
I'll second that.
#45
03/08/2003 (3:00 pm)
Quote:
Funny, my design for GORPE allows me to take 80% of the game and 100% of the game LOGIC to an entirely different engine but then again you skipped the part of the sentence where I said PLATFORM.

Where I was tought, platform came to mean the 'platform' you were developing on and the intended audience. In this case: Windows, Mac, Linux, and I still stand that TGE Script moves to each of these 'platforms' just as easily as Python does. But yes, I did ignore you when you suggested that Python script can be applied to different game 'engines' because it is entirely pointless to care about such a thing. I'm, and hundres like me here, are developing using the Torque engine. Why should Garage Games think twice about whether or not their scripting language they choose develop would/could be applied to a different engine? If a developer does care about such a thing, then I argue that Garage Game's time is better spent on more important issues, and that the developers themselves should do whatever they have to themselves to get their scripts to be cross-engine independent.


Quote:
Three letters for you "pyc". TGE script is compiled to bytcode which gets interpreted, so does Python except Python's compiler is far more mature and the language is more complete.

I fail to see how a general purpose scripting language could compile faster and more secure code than a scripting language designed specifically for the engine(meaning no extra features that the original designers of the engine thought was unnecessary and could possibly inadvertantly expose features of the engine or the os that the designers didn't want exposed).

Quote:
As for the cleanist backend to the engine. Have you actualy looked at what's reuired to expose and use C++ objects in TGE script as opposed to Python? Here's an exercise: Add a matrix vairable to the AIPlayer base class, let's call it "foobar" now expose this variable to both scripting languages. Now compare the amount of code you wrote to expose a new variable.

I count one line of C++ code.

Quote:
Next: Now go into the respective scripting languages and see how much code you write to manipulate that variable.

I count X amount of lines of script code to read and manipulate said variable. This depends on the complexity and amount of information stored in a variable. But assuming said variable was a matrix of size 2 x 2 then I count 4 lines of script code to read and begin manipulating said variable. One line each to extract the 4 pieces of information.


Quote:
And yet everyday we here of this engine or that engine switching to open languages. But this whle discussion is getting REALLY old. I cannot fathom why you insist on taking a stand against an establish general purpose language over a propritary pseudolanguage but frankly I don't care.

Maybe I'm behind the times, can you name these engines so that I may read more about them?

Quote:
I have to ask... Are you coming up with these rediculous rebuttals for enteratinment value alone or what?

I have my opinions and you have yours. But belittling me because you view my arguements as rediculous and invalid is an entirely different subject matter I do not wish to persue with you here.

My opinions are stated and I stand by them, as I'm sure you'll continue to stand by yours. But I would just rather have Garage Games make an official statement about this then continue this arguement. I respectively acknowledge your points and I will respectively decline to agree with them.
#46
03/08/2003 (4:06 pm)
I'm done arguing as well.. One small correction. I was taking issue with you left out half of the engine/platform statement and focused on the platform.

Bottom line is this: You don't HAVE to upgrade to using Python just like nobody HAD to upgrade from DOS.
#47
03/08/2003 (6:37 pm)
Moving on...

I agree that the single biggest improvement that we could do for Torque 2.0 is a refactoring to make it more modular (and the code cleaner). (6-8 weeks seems reasonable. I know nothing about scheduling.)

A good refactoring would lay the groundwork for language work, better graphics, etc. etc. etc. - without it, any such work is just hacks.
#48
03/08/2003 (7:37 pm)
I think you're missing one small piece of information... in order to even to have a static library and link to that, the scripting language has to be modularized. Previous attempts at separating the engine from the game revealed this.

Quote:Maybe I'm behind the times, can you name these engines so that I may read more about them?

I'll take this one. ;-)

Engines:
Crystal Space is an LGPL engine. I'm not sure if it's been used in any commercial projects yet.
The Nebula Device uses Tcl, Python, and Lua. It is freely available, and has been used in at least one commercial project.
OGRE is a rapidly-growing engine, quite neat. I know it at least uses Python, but I'm a bit out of date with it.

Commercial Games/Engines:
Lua has been used in: Baldur's Gate, Grim Fandango, MDK2, and Escape from Monkey Island (a SCUMM upgrade, I believe.) The list goes on and on.

Python was also used as a SCUMM replacement, there's a paper explaining the decision that gives even more examples of where Python's being used here. It was also used (and continues to be used) by Totally Games in concert with NetImmerse in Star Trek: Bridge Commander and their other projects.

And those are just the ones I can find/know off the top of my head. But really, just use whatever you're comfortable with. I think modularizing the existing scripting language will benefit both it, Torque, and the community.

As for requests, I think the first group of posts covered my main points. Once the engine gets separated out a bit, I think doing project-specific modifications will be trivial.
#49
03/09/2003 (8:24 am)
Lua is also used by Relic on the Homeworld franchise, at least the first two, ie Homeworld, and Cataclysm (I know, another studio did Cataclysm but they used relic's engine, didn't they ? 8p)
I dunno about Homeworld 2, or their other titles, but I wouldn't be surprised that they use it there too.
Nebula also had a Java binding at one point.
Falcon 4 used TCL/TK IIRC
Etc.

it's more than a trend, as the amount of work needed to have a full featured high level language ("script" sounds too pejorative sometimes :)), maintain it, etc. in itself explains why people are going for known quantities : after all, torquescript evolved over years to become what it is now.
Not sure it should be replaced altogether, or not, as I don't think I have all the data to make an informed judgement on this, but I certainly look upon making easier to plug in other languages as a big boon, if not a crucial feature...

(Part of the edit : and the reverse too : embedding Torque as a module, dynamic library, whatever for a high level language is not only cool from a geek pov, but also very interesting as a mechanism to extend the possibilities of game making with Torque. Hence why what Joshua is doing with PyTorque is so interesting, if not extremely important, imho)

Also part of the Edit : As for flexibility and modularity, I'm all for it within reason : it has to provide us with means to make better games easier, as in adding, extending, or taking out stuff we dont' require, replace it with something else, etc. I'm not too worried about the performance aspect, as I don't think anyone wants to shoot itself in the foot just for the sake of refactoring. To the contrary :making it easier to replace or drop in new functionality with also ease the optimizing tasks by giving you more room to work as in that it should be easier to control the ripple effects of your changes over the whole sourcetree.
And thing is, once you've settled on a design and have your framework in place, you can start optimizing stuff for your title, as some teams are doing right now.
#50
03/09/2003 (8:31 am)
To be a ghetto user...
TURRETS TURRETS TURRETS PLEASE!
#51
03/09/2003 (9:12 am)
Radiosity!

*wink*

-Josh Ritter
Torque Evolutionary
#52
03/09/2003 (9:27 am)
Tim, turrets are not an engine feature.

About scripts, as long as any required framework goes in to support scripts I'll be happy. I believe that the .cs advocates will 'see the light' when they use a more mature scripting language for Torque.

How about some more ideas?

1. Replace all C style casts with appropriate C++ style casts (static_cast, reinterpret_cast, const_cast, dynamic_cast). The advantages here are that you can search for them, and they have some type safety (varies).
2. Debug memory leak detection. This can range from simple (just telling you that an object of size x or type y leaked) to complex (telling you which line it was allocated from, and the call stack).
3. Remove the platform types (F32, S32, U32, etc.) except where they actually matter. Maybe I don't get it, but it seems to me that having all of these types everywhere would make it more difficult to port to 64 bit, since instead of using the most efficient data type, it would use a 32 bit one. Clearly these are still needed in places like networking.

These are just my ideas, I know you people have things to add! Speak up, we need a list =)

edit: My whole point is to make the engine flexible and pleasant to work with.
#53
03/09/2003 (10:42 am)
David,

You have to keep the platform types like F32 around exactly FOR the reson you mentioned. If you go to a 64 but platform and the default number is 64 bits wide instead of 32 you end up with incom[atible binary files and netowrk prorotcols if everyone is using native number. Instead you use things like "F32" which say this is a float and it is guarnteed to be 32 bits wide.
#54
03/09/2003 (10:58 am)
Lua is a nice and small, relatively fast scripting language...

I have experience with Lua bindings... it's binding mechanism is very simple... to make a Lua binding to the Torque console system would be pretty easy...

The problem is all language bindings to the Torque console system kind of run into the console system.. specifically everything being a string... meaning, like the C++ code ... Lua, Python, etc, script has to:

1. Be knowledgeable about what it is being passed
2. Do the conversion

This get's clunky very fast... the console system needs type information for this to work really well... The Nebula Device's script abstraction is quite good and has type information... definately something to look at...

It also a scenegraph with OpenGL and D3D8/9 support(with shaders)... hm

-J
#55
03/09/2003 (12:01 pm)
Turrets are the latest feature every engine must have!... they were so much fun and worked so good in t2. I would really appreciatte having to learn python instead of torque script. not that torque script is unfathonable.
#56
03/09/2003 (6:02 pm)
J. Donavan Stanley,

Did you, um, read what I wrote? Your response is a little confusing.

Dave:
Quote:
Remove the platform types (F32, S32, U32, etc.) except where they actually matter. ... Clearly these are still needed in places like networking.

J. Donavan:
Quote:
...you end up with incom[atible binary files and netowrk prorotcols if everyone is using native number. Instead you use things like "F32" which say this is a float and it is guarnteed to be 32 bits wide.

I would consider networking and file loading as things that actually matter. For normal internal processing however, native types are preferable for performance reasons.
#57
03/09/2003 (6:23 pm)
Sorry David it was a knee jerk response to seeing removal and platform types in the same sentenace together.
#58
03/10/2003 (9:26 am)
So whats the Chances any of these suggestions actually get considered for implementation?

Matt
#59
03/10/2003 (10:16 am)
coded fairly easy, imp'ed that would be a GG type of question.

I am sure their is enough coding skill/knowledge in the GG community to knock out most if not all of the topics posted. The trick is getting GG to step up to the plate and find time to add the to CVS. This is in no way a slam against GG, their buckets have been overflowing for a considerable amount of time.

-Ron
#60
03/11/2003 (12:26 am)
My wishlist would be..

1. rendering code seperated and abstracted, making it more platform independent (no, not all platforms support opengl.. just think about the xbox or ps/2).
Having the ability to use different renderers, like one renderer would be for wireframe rendering.. another for non photo realistic rendering etc.
Also having rendertargets, making it possible to render to pbuffers (or other api-equivelant) so it would be possible to do stuff like shadow buffers and reflecting objects (not just flat mirrors)

2.
graphic shaders: to define how a surface looks
sound shaders: to define how a sound is played
materials: to define how a material looks (uses graphic shader), sounds(uses sound shader) and behaves when for example it's hit (physics shader???)..

3. seperating the functionality of terrain, skybox, interiors more.. seems to be tied together too much

4. streaming geometry/terrain? making it possible to have huge worlds that don't have to be completely in memory all at once..

5. move -all- the game specific code to the scripting language..


Recently i came to the conclusion that it's in everyone's best interest if everyone would share their work on torque (as long as it's not too game specific)
why?

1. it's harder for us to compete with full scale, fulltime professional development teams than it is to compete with eachother.
2. we're all re-inventing the wheel all the time.
3. the torque engine is unfinished, there are still a lot of things that need to be done like the sound (buggy), the water (swimming) etc.
if you make radical changes, you won't be able to update your code with the new changes from the cvs, and you need to fix all that by yourself.. unless you share the code and (if it's good enough) it'll be included in the shared code base..
4. there seems to be this fear between developers to share their code.. as if the code they have will make the difference between having a succesfull game and an unsuccesfull game.
well.. i disagree.
when it comes to gameplay code, yes.. it may make a difference.. but the other code? considering you'll have a lot less resources if you do it by yourself (even with a small team) you probably won't be able to implement that much in a given ammount of time, compared to a situation where we all share our work.
i like to think that's it's HOW you use your technology, not just having the technology by itself.

ofcourse it's easy to say that everybody should share and not share yourself...
i'm currently working on rewriting the rendering code
my plan is to release it as a code snippet when enough of the work has been done..

i don't really care if other people would use my code because i'd like to think that i'm able to make a really good game..
if i didn't believe that.. why would i be wasting my time on this anyway? :)
oh yea, it's lots of fun too :)
but that's not a reason -not- to share ;)