Game Development Community

Transparent Development, Torque 2, and You!

by Stephen Zepp · 10/18/2007 (5:26 am) · 122 comments

There has been some buzz floating around since IGC this year, and a lot of it has been around InstantAction, but we also announced a couple of additional topics that are starting to sink in, so I wanted to dive in to the deep end of the community pool and make some waves....

Transparent Development farm3.static.flickr.com/2031/1610605284_c4d5630c8d_m.jpg
Sounds nifty, but what is it? One analogy I've heard that is a pretty solid description is consider TD a window into the offices of the development staff working on Torque 2--see what's being worked on this week, read discussions about designs as they are being discussed internally, and most importantly be able to discuss the designs and implementations themselves, as they occur.

We're not talking marketing information for a product that has already been developed--we're talking watching, and even being able to discuss and provide feedback on the development of Torque 2 as it occurs. I'll get to Torque 2 in a moment, but I want to reinforce that this is a fundamental shift back to the way GarageGames used to interact with the community--away from keeping completely silent regarding what we're working on and why, and back to giving the community rapid and fully packed pages of information on the development as it occurs--not after the fact, and especially not after it's released. Torque 2 isn't even scheduled for beta until well in to 2008, but we're working as rapidly as we can to get infrastructure in place to set up discussions as the engine team works on development.

We're talking roadmaps, milestone lists, design write ups, expected work flow and implementation standards, the works--all up front, all the time. One of the projects that I am working on right now is designing a new set of forums, wiki, and related web tools to manage extensive and extended data flow between the Transparent Development Community (that's you guys) and the engine development team (that's us!). Once this infrastructure is in place, we're going to open it up for communication purposes to a limited set of users for beta testing for a few weeks, and then we will make the information available to all.

[edit: see below in the comments section for further clarification on content and timing]

It's going to be several weeks yet before this infrastructure is ready, and it will be a few weeks of beta testing of that infrastructure as well before we're ready to turn on the fire hose, so please be patient while we get things set up for this to work in the long term--when it's ready, we'll let everyone know and let the show begin!

farm3.static.flickr.com/2114/1609890839_c49211e6b6_t.jpgI do want to make something fundamentally clear from the beginning (and trust me, I'll keep pressing this point until it sinks in fully!): Torque 2 development will be an iterative process, very similar to the SCRUM, or Agile/Spiral Development models--and one of the key aspects of this style of development is that sometimes you go down a path, and it turns out that the path is not the best solution. When this occurs, the best course of action is simply to cut out the results of that development spiral, and reset your project expectations. Sometimes this means removing features, adjusting the milestone list, or delaying (or in some cases even canceling) entire systems.

It is critical that the entire Transparent Development Community understand that this development process you will be watching and discussing is a fluid one--and it should be fully expected that features and systems being discussed one week may very well end up being cut forever the next. No projects, games, and especially no funds or contracts should ever be committed during this development process--if you need to make commitments or financial decisions regarding your projects, then the Torque 2 Transparent Development Community is absolutely not somewhere you should be.

Torque 2

So what's Torque 2? To put it simply, Torque 2 is a ground up re-factor of the entire architecture of all of our engines into a new engine with some pretty amazing architecture and features. We've learned quite a bit over the last several years of engine development, and the time has come to take those lessons and build a new engine that not only provides the maximum flexibility possible to game developers, but fundamentally change many of the designs of the engine itself--correcting design assumptions and work flows that came about over the years that we ourselves find too restrictive, and along the way providing customization and feature enhancement capabilities that were never available before.

farm3.static.flickr.com/2215/1610729962_0a412b7a4f_m.jpgTorque 2's fundamental changes fall under three primary areas:

Modularity of Systems: In the past, it was very difficult to leverage into Torque systems, libraries, and raw code written by 3rd parties, such as ODE/Havoc/PhysX Physics, alternate scripting languages such as Lua, Python, and even .NET languages, and many other cool things that are available to game developers. Torque is extremely integrated, with each of the systems tightly coupled to provide a powerful, but ultimately restrictive engine when it comes to "mashing up" other solutions into a coherent whole--and Torque 2 is designed from scratch to correct those issues by allowing game developers to replace major system modules in a "plug and play" style--as long as the external systems match interfaces that the Torque 2 framework expects, you will be able to use just about any 3rd party (or self-developed) system within the core Torque 2 framework.


Abstraction and Interfaces: One of the key ways we will be able to provide this type of "swap out" modularity is by abstracting most of the core systems via the use of interfaces. Each major system will describe a set of interfaces that other systems and objects will use to communicate with the system, so instead of calling code directly within a system, external modules and objects will interact with the interfaces that system exposes--allowing other systems to replace core modules at the whim of the developer.

Torque 2 has already leveraged the previously developed GFX abstraction layer present in TGE-A, and in addition has had several iterations of abstraction of core systems as well, such as the sound layer (now abstracted through a new SFX system), lighting (fully abstracted into a separate module), collision and physics (currently under iterative refactoring towards a comprehensive interface), and many others. As the iterative development spirals of Torque 2 continue, additional systems will be abstracted and interfaces implemented for even more comprehensive modularity.

farm3.static.flickr.com/2307/1609952371_065c473022_m.jpgComponents: The concept of a component based architecture is actually very simple--but it has some far reaching design implications, and in many cases can take some time to adapt towards. In most game engines, including Torque, you have a very deep set of classes that form a pretty static ancestry, and you use inheritance of all of these classes in your game object level classes. Unfortunately, this has several issues that become more apparent the longer you use these classes for development, especially across multiple games: your game object classes start becoming more and more bloated, and less appropriate for specific uses. Anyone that takes a good look at the Torque ShapeBase class will soon see what I mean--there are still left over methods in the ShapeBase class that are part of original Tribes code, and aren't particularly useful for most games.



farm3.static.flickr.com/2373/1610605460_2ac1c2feae_m.jpgHere's an example of exactly what I mean: Back when BraveTree was working on their environment pack, one of the neat things they wanted to have was butterflies with random movement about the scene. Pretty nifty stuff--they derived a new class off of ShapeBase, and their butterflies merrily wandered about the stronghold mission randomly--until one of them happened to run in to a crossbow object placed in the scene.

farm3.static.flickr.com/2271/1609717845_b61ed13c49_m.jpgSuddenly, this tiny little butterfly picks up this huge crossbow, and starts swinging it back and forth madly as it continues to wander about--merrily meandering about the scene with a totally unexpected weapon of mass destruction dangling from its cute little mandibles.

It was immediately obvious what was going on, and as soon as they picked themselves up from off the floor and stopped laughing, they realized that since the butterfly inherited from ShapeBase, it also inherited the ability to collide with Items, and mount weapons--and that's exactly what it did!

So what does this have to do with components? Well, with the idea of components (based on the Object Oriented Programming concept of aggregation, sometimes called composition), instead of inheriting a series of ever more complex functionalities that your particular object may not need, you create very small units of functionality that accomplish simple tasks--and "build" your game objects from these little components. In the case of the butterfly, they might have added a render3D component, a collision3D component, a hoverFlightMovement component, and a AIRandomMove3D component--and thereby had all the functionality they needed for the butterfly, without any of the bloat that inheriting from a full class would have given them.

We'll be covering components in quite a bit more depth once the Transparent Development Community is in place, but if you are interested in learning more about how component based game development works, TorqueX also uses components, and is a great place to learn how the new design concepts can be used.

What this means for You

Well, the very first thing it means is not a lot to be honest--this is an announcement of a new development style (transparency), and a new engine that is just now entering heavy development. Torque 2 is not going to be ready for use in a major (or even minor) game project until sometime in 2008 (more accurate time line will come when we get more iteration cycles finished up). If you are currently working on a game with Torque (or another engine), or are planning on starting a new project in the next 6 to 9 months, Torque 2 is not for you at this time.

If you are someone that is interested in how an engine is developed, and want to participate in discussions of that development process as it occurs, stay tuned for further announcements about the Transparent Development Community. Once it opens to the public stop on by and hang out--at the least it should be educational and informative, and hopefully some of you might even generate ideas, designs, and implementations that may very well be factored in to the future of Torque.

I'm already well over my word count for a .blog, so I'll answer specific questions in the comment area below.
Page «Previous 1 2 3 4 5 6 7
#1
10/18/2007 (6:23 am)
Some additional notes on Torque 2:

  • *Not intended to be backwards compatible with current Torque Engines.
  • *Includes both 2D (from TGB) and 3D (TGE-A) capabilities
  • *New directory and system structure that separates engine, application (game), and tools (external apps)
  • *Multiple components can provide different implementations for the same task--for example, you may have a dtsLoaderComponent, a colladaLoaderComponent, and a superAwesomeNewFormatLoaderComponent all on different objects in your scene
  • *poly soup collision will be the primary collision mechanism
  • [li]**BSP is not currently scheduled as a collision/rendering system, but the modular nature of systems and components does not preclude it for the future
  • *Torque 2 networking will be modular as well (allowing for replacement modules), and the initial module will be based upon our Torque Networking Library initiative. Other modules (and their related components) may be provided towards the end of the first release phase by GG, community members, or provided by 3rd parties.
  • *TorqueX as a product will effectively be unaffected by Torque 2 development and productization.

TGE, TGE-A, and TGB (as well as TorqueX) will continue to receive active support until Torque 2 is well established as a development engine. At that time (well in to 2008), we will begin shifting developers towards using the new engine for future projects.

While TorqueX is an excellent way to get an early glimpse of the component architecture, Torque 2 is written in C++, and TorqueX is in C#, so there is no direct code to code relationship between the two engines.
#2
10/18/2007 (6:38 am)
Wow! Stephen best product writeup ever! Good work!
Ps. Sign me up on the beta list, send over a NDA asap :)
#3
10/18/2007 (6:51 am)
Stephen must have forgotten to mention - that picture of the guy with the mask in the water - that's the punishment when one of us breaks the nightly build.

[Nice writeup Stephen!]
#4
10/18/2007 (7:01 am)
The writeup is excellent, but more importantly the content of the post is absolutely huge.

I'm going to have to re-think my plans a bit... finally, a real game engine framework.

edit - removed NDA comment :D
#5
10/18/2007 (7:06 am)
Guys, don't worry about NDA's--we're simply talking about beta testing the web infrastructure, not Torque 2 itself--and it's not going to be a very long period--we just want to make sure the new forums and related services work as expected, and don't have any speed bumps before we start making the transparent development process available to everyone.
#6
10/18/2007 (7:45 am)
Two questions about Torque2 : will cross-platform be on the priority list ? And will the code remain open and customizable for owners ?

Doing cross-platform development, I know it is harder to develop, but it helps to track bug (some bugs that don't show on one platform suddenly appear on another platform because of little/big endian coding for instance) and make the code cleaner.

And secondly, I choose TGE one year and a half ago because of it's cross-platform capabilities (and its network layer of course).

Nicolas Buquet
http://www.buquet-net.com
#7
10/18/2007 (7:59 am)
I was just about to comment on cross platform, good timing!

Cross platform capability is, as it always has been, important to us, and the underlying architecture is being written with cross platform compatibility built in as it always has.

We realize this is an area of large importance to quite a few developers, and we also realize that our implementation priorities are not always the same as some community members, so I want to be right up front from the beginning:

We plan on testing Mac compatibility all the way through the development process, as much as possible. In fact, we recently hired a second engineer with extensive Mac experience (John Hearn) to work alongside Paul Scott to make sure our builds stay in touch with the Mac OS.

Built in ability to be linux compatible is also an implementation goal, but build to build linux testing is not on the current task list of any of the developers working actively with the engine. It's possible that we may outsource linux compatibility implementation, perform it internally from time to time, or allow the Transparent Development Community or a limited set of developers within that community to support various versions of linux compatibility during a (hypothetical at this point) closed beta period, but none of these options are set in stone, and any/all are subject to change.

I ask very strenuously that if you have specific reactive comments to the above, that you either post them in a forum thread, or wait until the TD Community is stood up--I'd rather not have this thread become an OS or cross-platform debate. Let's keep it focused on questions and answers, instead of debates about those answers please :)
#8
10/18/2007 (8:03 am)
Wow, the "new" GarageGames and their public process is brilliant, really! Can I sign up somewhere as beta tester / associate something? ;-)

@Nicolas: I'm pretty damn sure it will be multi platform. As it seems to me, T2 will be the new platform for all games and IA in the future and I'm pretty sure that they (GG) don't want to loose the ability to port even their own games to WII / PS2/3 / XBox / Mac and alike, that would require multi platform code. And from what Stephen said, it will be better customizable than any other Torque engine before because the code gets a new leightweighted structure and clearly seperated into single components. Personally I bet that it will be in future easier (!) to get into the code than before.

@Stephen: Awesome blog! (and if you haven't already, go to bed now, it's damn late in Oregon according to the world time map... ;-)
#9
10/18/2007 (8:46 am)
Couple related queries on components and the networking model, presuming the questions are not premature:

1 - is a masking system similar to the current itterations still going to be embeded, or is that being refactored on out at some point?

2- are you leaning towards oop or aop when it comes to packet-fill? (oop being filling out this object's data then the next, ect ect. till you hit packesize and try again next time, aop being filling out critical data first, like say object transform data for all object, then health, names being a distant non-gamebreaker ect ect)
#10
10/18/2007 (8:53 am)
Very exciting times \o/

Thanks for the writeup Stephen ^^

PS : Are TGB Behaviors the same thing (or almost the same) as components ?
#11
10/18/2007 (9:21 am)
Thanks for your answers and good luck for the (very) near future.
#12
10/18/2007 (10:32 am)
Sounds like Torque 2 is going to be a major product when it's finished and I for one will certainly relish the modularity that you guys are building in. Great write up Stephen.
#13
10/18/2007 (11:53 am)
This sounds fantastic. :D Exciting times ahead I think.
#14
10/18/2007 (11:58 am)
Great .blog post Stephen!
One point I'm interesting in...
Do you guys plan to make an engine to be "localization-ready"? Something like adding support for localization for every single part, be that Editors (GUI/World) or the game itself.
And thanks for good response on my Unicode-support question during IGC/Round-Table, but it was not fully covered..
#15
10/18/2007 (12:17 pm)
It[Modularity] appears to me, this is how the 'future' of the 3D gaming +/- online experience should be developed. Modularity is a very forward thinking approach to this 'industry'. Just as I am able to 'customize' my 'ride' or 'homeImprovement' or 'lifeStyle', enabling the endUser/consumer to decide how to assemble things to their 3D/gaming specs is appreciated and will hopefully be rewarded with a fair market share to the creator/maintainers. I mean, I feel that game 'mechanics/gamePlay' are already reaching an 'endPoint', and what is needed is a fresh 'look' to product. Meaning, most 'projects' want some very basic starting points of commonality...good physics, graphicsRendering, etc... and being able to provide the 'spare parts' to hammer together 'stuf'....enabling the developer to create the world of the dreams is exactly what the Dr. ordered...

"...the future is here; Today!".
#16
10/18/2007 (1:33 pm)
Just two questions about Torque 2:

- There will be a documentation? Cause the current torque documentation is very incomplete.

- There will be changes in the waterblock? I mean, shore and underwater environment.

Thanks
#17
10/18/2007 (1:34 pm)
Quote:TGE, TGE-A, and TGB (as well as TorqueX) will continue to receive active support until Torque 2 is well established as a development engine.

I noticed you included TorqueX in the above statement and am curious as to why? I understand if development stopped on TGE, TGE-A, and TGB, but why would you stop supporting TorqueX? It seems that TorqueX and Torque2 would be both supported with the others going away?

.net and C# in particular becomes more and more usable in the gaming industry every year, so replacing it with yet another C++ engine, no matter how good that engine is, just doesn't seem like a good business decision.
#18
10/18/2007 (1:49 pm)
@Jonathan: If I understood Stephen right, then you could easily interface T2 also from C#, like from any other (compatible) language. So the "union" of all engines will be able to be interfaced from nearly everywhere. In theory you could even use JavaScript for it. Stephen, please correct me if I'm wrong.
#19
10/18/2007 (1:55 pm)
Thanks for the write up. This is good to hear especially for those of us who have no idea how things like this are created. Thanks for the free education. :)
#20
10/18/2007 (1:58 pm)
Stephen, thanks for that awesome blog! It was very informative.

Also, do you know if GG has any plans to hire/re-task even 1 person to work on the Linux side of T2? It seems, at least to me, that if GG wants to support Linux in the future that it would save GG money in the long run to spend the money early in development rather than trying to port it after the fact.
Page «Previous 1 2 3 4 5 6 7