Game Development Community

Transparent Development, Torque 2, and You!

by Stephen Zepp · 10/18/2007 (5:26 am) · 122 comments

There has been some buzz floating around since IGC this year, and a lot of it has been around InstantAction, but we also announced a couple of additional topics that are starting to sink in, so I wanted to dive in to the deep end of the community pool and make some waves....

Transparent Development farm3.static.flickr.com/2031/1610605284_c4d5630c8d_m.jpg
Sounds nifty, but what is it? One analogy I've heard that is a pretty solid description is consider TD a window into the offices of the development staff working on Torque 2--see what's being worked on this week, read discussions about designs as they are being discussed internally, and most importantly be able to discuss the designs and implementations themselves, as they occur.

We're not talking marketing information for a product that has already been developed--we're talking watching, and even being able to discuss and provide feedback on the development of Torque 2 as it occurs. I'll get to Torque 2 in a moment, but I want to reinforce that this is a fundamental shift back to the way GarageGames used to interact with the community--away from keeping completely silent regarding what we're working on and why, and back to giving the community rapid and fully packed pages of information on the development as it occurs--not after the fact, and especially not after it's released. Torque 2 isn't even scheduled for beta until well in to 2008, but we're working as rapidly as we can to get infrastructure in place to set up discussions as the engine team works on development.

We're talking roadmaps, milestone lists, design write ups, expected work flow and implementation standards, the works--all up front, all the time. One of the projects that I am working on right now is designing a new set of forums, wiki, and related web tools to manage extensive and extended data flow between the Transparent Development Community (that's you guys) and the engine development team (that's us!). Once this infrastructure is in place, we're going to open it up for communication purposes to a limited set of users for beta testing for a few weeks, and then we will make the information available to all.

[edit: see below in the comments section for further clarification on content and timing]

It's going to be several weeks yet before this infrastructure is ready, and it will be a few weeks of beta testing of that infrastructure as well before we're ready to turn on the fire hose, so please be patient while we get things set up for this to work in the long term--when it's ready, we'll let everyone know and let the show begin!

farm3.static.flickr.com/2114/1609890839_c49211e6b6_t.jpgI do want to make something fundamentally clear from the beginning (and trust me, I'll keep pressing this point until it sinks in fully!): Torque 2 development will be an iterative process, very similar to the SCRUM, or Agile/Spiral Development models--and one of the key aspects of this style of development is that sometimes you go down a path, and it turns out that the path is not the best solution. When this occurs, the best course of action is simply to cut out the results of that development spiral, and reset your project expectations. Sometimes this means removing features, adjusting the milestone list, or delaying (or in some cases even canceling) entire systems.

It is critical that the entire Transparent Development Community understand that this development process you will be watching and discussing is a fluid one--and it should be fully expected that features and systems being discussed one week may very well end up being cut forever the next. No projects, games, and especially no funds or contracts should ever be committed during this development process--if you need to make commitments or financial decisions regarding your projects, then the Torque 2 Transparent Development Community is absolutely not somewhere you should be.

Torque 2

So what's Torque 2? To put it simply, Torque 2 is a ground up re-factor of the entire architecture of all of our engines into a new engine with some pretty amazing architecture and features. We've learned quite a bit over the last several years of engine development, and the time has come to take those lessons and build a new engine that not only provides the maximum flexibility possible to game developers, but fundamentally change many of the designs of the engine itself--correcting design assumptions and work flows that came about over the years that we ourselves find too restrictive, and along the way providing customization and feature enhancement capabilities that were never available before.

farm3.static.flickr.com/2215/1610729962_0a412b7a4f_m.jpgTorque 2's fundamental changes fall under three primary areas:

Modularity of Systems: In the past, it was very difficult to leverage into Torque systems, libraries, and raw code written by 3rd parties, such as ODE/Havoc/PhysX Physics, alternate scripting languages such as Lua, Python, and even .NET languages, and many other cool things that are available to game developers. Torque is extremely integrated, with each of the systems tightly coupled to provide a powerful, but ultimately restrictive engine when it comes to "mashing up" other solutions into a coherent whole--and Torque 2 is designed from scratch to correct those issues by allowing game developers to replace major system modules in a "plug and play" style--as long as the external systems match interfaces that the Torque 2 framework expects, you will be able to use just about any 3rd party (or self-developed) system within the core Torque 2 framework.


Abstraction and Interfaces: One of the key ways we will be able to provide this type of "swap out" modularity is by abstracting most of the core systems via the use of interfaces. Each major system will describe a set of interfaces that other systems and objects will use to communicate with the system, so instead of calling code directly within a system, external modules and objects will interact with the interfaces that system exposes--allowing other systems to replace core modules at the whim of the developer.

Torque 2 has already leveraged the previously developed GFX abstraction layer present in TGE-A, and in addition has had several iterations of abstraction of core systems as well, such as the sound layer (now abstracted through a new SFX system), lighting (fully abstracted into a separate module), collision and physics (currently under iterative refactoring towards a comprehensive interface), and many others. As the iterative development spirals of Torque 2 continue, additional systems will be abstracted and interfaces implemented for even more comprehensive modularity.

farm3.static.flickr.com/2307/1609952371_065c473022_m.jpgComponents: The concept of a component based architecture is actually very simple--but it has some far reaching design implications, and in many cases can take some time to adapt towards. In most game engines, including Torque, you have a very deep set of classes that form a pretty static ancestry, and you use inheritance of all of these classes in your game object level classes. Unfortunately, this has several issues that become more apparent the longer you use these classes for development, especially across multiple games: your game object classes start becoming more and more bloated, and less appropriate for specific uses. Anyone that takes a good look at the Torque ShapeBase class will soon see what I mean--there are still left over methods in the ShapeBase class that are part of original Tribes code, and aren't particularly useful for most games.



farm3.static.flickr.com/2373/1610605460_2ac1c2feae_m.jpgHere's an example of exactly what I mean: Back when BraveTree was working on their environment pack, one of the neat things they wanted to have was butterflies with random movement about the scene. Pretty nifty stuff--they derived a new class off of ShapeBase, and their butterflies merrily wandered about the stronghold mission randomly--until one of them happened to run in to a crossbow object placed in the scene.

farm3.static.flickr.com/2271/1609717845_b61ed13c49_m.jpgSuddenly, this tiny little butterfly picks up this huge crossbow, and starts swinging it back and forth madly as it continues to wander about--merrily meandering about the scene with a totally unexpected weapon of mass destruction dangling from its cute little mandibles.

It was immediately obvious what was going on, and as soon as they picked themselves up from off the floor and stopped laughing, they realized that since the butterfly inherited from ShapeBase, it also inherited the ability to collide with Items, and mount weapons--and that's exactly what it did!

So what does this have to do with components? Well, with the idea of components (based on the Object Oriented Programming concept of aggregation, sometimes called composition), instead of inheriting a series of ever more complex functionalities that your particular object may not need, you create very small units of functionality that accomplish simple tasks--and "build" your game objects from these little components. In the case of the butterfly, they might have added a render3D component, a collision3D component, a hoverFlightMovement component, and a AIRandomMove3D component--and thereby had all the functionality they needed for the butterfly, without any of the bloat that inheriting from a full class would have given them.

We'll be covering components in quite a bit more depth once the Transparent Development Community is in place, but if you are interested in learning more about how component based game development works, TorqueX also uses components, and is a great place to learn how the new design concepts can be used.

What this means for You

Well, the very first thing it means is not a lot to be honest--this is an announcement of a new development style (transparency), and a new engine that is just now entering heavy development. Torque 2 is not going to be ready for use in a major (or even minor) game project until sometime in 2008 (more accurate time line will come when we get more iteration cycles finished up). If you are currently working on a game with Torque (or another engine), or are planning on starting a new project in the next 6 to 9 months, Torque 2 is not for you at this time.

If you are someone that is interested in how an engine is developed, and want to participate in discussions of that development process as it occurs, stay tuned for further announcements about the Transparent Development Community. Once it opens to the public stop on by and hang out--at the least it should be educational and informative, and hopefully some of you might even generate ideas, designs, and implementations that may very well be factored in to the future of Torque.

I'm already well over my word count for a .blog, so I'll answer specific questions in the comment area below.
#21
10/18/2007 (2:01 pm)
Quote:.net and C# in particular becomes more and more usable in the gaming industry every year, so replacing it with yet another C++ engine, no matter how good that engine is, just doesn't seem like a good business decision.
Managed code is not going to replace anything in the "true" game industry anytime soon (and even not soon). C# and .net is just something Microsoft is pushing for hobbyists wishing to make games for 360, it was never intended to support full blown multi-platform engines like Torque 2, it will never be suited for that.
#22
10/18/2007 (2:20 pm)
"Managed code is not going to replace anything in the "true" game industry anytime soon"

I think you have to reconsider that statement soon :)
#23
10/18/2007 (2:21 pm)
I agree. Read an article here about some retail games that already were build fully on managed code.
#24
10/18/2007 (2:23 pm)
C# is not a candidate for replacing C++. They are both tools and they both have their uses. One is not better than the other, they are different. It's literally that simple.

Can we please not have a C++ vs. C# war here please?
#25
10/18/2007 (2:51 pm)
Will T2 offer anything new platforms such as mobile devices like the DS or PSP?
#26
10/18/2007 (3:21 pm)
I love the Transparent Development idea. However, I am pleading with everyone else in the community to be realistic about this. Professional software development is very inexact. As Stephen says, features fully discussed/designed one week may hit a major roadblock and be entirely cut the next week. This is a reality in software development. I will put any amount of money right now on a bet that T2 will not hit every deadline and milestone feature list.

So, bottom line to everyone "out there:" complaining, ranting, and insulting people when a feature gets cut or a deadline is pushed (because it will happen) does nothing for anyone. So, let's all be good "stewards" of the responsibility GG is giving us (as a community) here.
#27
10/18/2007 (3:29 pm)
Nothing left to say. Can't wait until we can get a look at it. :-)
#28
10/18/2007 (4:10 pm)
I've been waiting for this for a long time :D

What will happen to TGE/TGEA/TX?
#29
10/18/2007 (4:10 pm)
@Mark Dynna
I support your pleas here. I can only imaging there will be at least one person who will not agree with what is going. I hope that the communities/supporters of GG maintains the higher more mature standard. In other words ignore those who don't have valid comments. If you have something negative to say just make sure you support and document it with good sense. :)
#30
10/18/2007 (4:29 pm)
Ok, lots of really good questions! I'm going to quick answer the ones that are known, and defer the ones that are future design oriented until the TDC (Transparent Development Community) is up and running.

Quote:
Couple related queries on components and the networking model, presuming the questions are not premature:

1 - is a masking system similar to the current itterations still going to be embeded, or is that being refactored on out at some point?

2- are you leaning towards oop or aop when it comes to packet-fill? (oop being filling out this object's data then the next, ect ect. till you hit packesize and try again next time, aop being filling out critical data first, like say object transform data for all object, then health, names being a distant non-gamebreaker ect ect)

First, let me say that networking will be just another module--if you need peer to peer for your networking system, or possibly DIS (military networking protocol), or something unusual--it's just another module.

1) Yes, we will continue to use something similar to our existing masking system. Currently, we will be folding TNL (Torque Network Library, which is effectively a "1.5" version of the core networking within our existing engines) into Torque 2 in some form (componentized and modular based of course)

2) I can't say that it's been addressed. Off the top of my head, I see a lot of pros and cons. Great topic for the TDC once it's up and running!

Quote:
PS : Are TGB Behaviors the same thing (or almost the same) as components ?

TGB Behaviors are close, in use specifically (although not in implementation really). The underlying implementation is quite a bit different since we had to be fully backwards compatible, but in general if you like behaviors, you'll love components.

Quote:
Do you guys plan to make an engine to be "localization-ready"? Something like adding support for localization for every single part, be that Editors (GUI/World) or the game itself.
And thanks for good response on my Unicode-support question during IGC/Round-Table, but it was not fully covered..

UniCode is very much on the list of things to be more successful with in Torque 2, but it's not on the first several levels of iteration tasks lists as far as I am aware. I agree with you very much however that it is both very important, and needs to be treated more holistically. We are planning to move to a String class implementation for a variety of reasons, and it should help to make Unicode support less difficult.

Quote:
Just two questions about Torque 2:

- There will be a documentation? Cause the current torque documentation is very incomplete.

This has been discussed in several places (unrelated to Torque 2 itself), but yes, we will be very much focused on all of the additional aspects besides simply pushing an engine out. I'm going to talk a bit about development versus productization at the bottom of this post, which relates to documentation as well.

Quote:
- There will be changes in the waterblock? I mean, shore and underwater environment.

This is a perfect example of underscoring the differences between engine, application (game), and tools--waterblocks shorelines, underwater environments--all of those functionalities have absolutely nothing to do with the engine--they are all part of the application. In other words, they are game level objects, not engine architecture.

The short answer is "no, at this time, there are no tasks or iterations related to waterblocks, shorelines, underwater development, or any other game level object functionalities--there may be in the future, but we're at the architecture refactoring stage, not the example application stage".

The alternate short answer is "these type of functionalities are suited for game object components, not the engine. At a minimum we will most probably have example components that provide the same level of functionality that exists in, for example TGE-A today. The benefit of the underlying architecture change is that it will be much less difficult to add things like shoreline effects, underwater effects, and other game level systems quickly and easily with components.

Quote:
I noticed you included TorqueX in the above statement and am curious as to why? I understand if development stopped on TGE, TGE-A, and TGB, but why would you stop supporting TorqueX? It seems that TorqueX and Torque2 would be both supported with the others going away?

Was a "catch-22" situation--if I didn't say Torque X, someone would ask about it--if I did, someone would read it differently! Not a big deal at all, I just should have spent some time being more clear from the beginning.

TorqueX and Torque 2 are completely unrelated, and are two completely different engines on different underlying technologies. TGE, TGE-A, and TGB are all in some way affected by the long term evolution of Torque 2, but Torque X is not, at all. Torque X focused users can effectively ignore just about all of the Torque 2 discussions--it won't affect you in any meaningful way.

Quote:
@Jonathan: If I understood Stephen right, then you could easily interface T2 also from C#, like from any other (compatible) language. So the "union" of all engines will be able to be interfaced from nearly everywhere. In theory you could even use JavaScript for it. Stephen, please correct me if I'm wrong.

Close--you could write a new script module that uses C# (or any .NET language--or any other language) as your scripting language. You could theoretically at least treat TorqueX as a module within Torque 2 (I'm reaching here, there might be some technical reason why this would be difficult, but I'm not aware of it at this time if there is one), but there would be no real benefit in this particular case.

Yes, you could certainly use JavaScript as your scripting language, if you (or someone else) write the module for it.

Quote:
Also, do you know if GG has any plans to hire/re-task even 1 person to work on the Linux side of T2? It seems, at least to me, that if GG wants to support Linux in the future that it would save GG money in the long run to spend the money early in development rather than trying to port it after the fact.

I covered this about 7 comments down, but to summarize: It's certainly possible. It's not currently on the actions list for Torque 2 at this time.

Quote:
Will T2 offer anything new platforms such as mobile devices like the DS or PSP?

I can't state any specific platforms at this time, but I think a reasonable conclusion could be made that of course it's in both our, and your, best interests for Torque2 to be as cross-platform as possible, with as many target platforms as feasible. The two specific platforms you bring up have not been discussed to my knowledge, but it's public knowledge that we already have an XBox 360 platform layer, as well as the Torque for Wii code base, and while I'm not saying it flat out, it's reasonable to assume that they could make appearances in some form in Torque 2 well in the future.

Ok, one topic that we haven't covered yet, and one that I can't go into detail too deeply in (moves away from transparent development, and in to competitive business strategy), but it's important to note that the Transparent Development Community will be completely focused on engine development, not product development. It sounds like two names for the same thing, but that's only because in the past, we've never actually done much work on product development--we've just finished an engine to a release-able point, and pushed it out the door with a couple of getting started/how-to docs and called it good.

With the resources we now have available after the IAC partnership, we have stood up an entire team who's sole focus is providing a cohesive product to our customers--tools, documentation, tutorials, examples, demos, and related materials to make working with Torque 2 a well thought out process instead of a "jump in and sink or swim" type of experience.

We won't be going into much detail on this aspect in the short or mid term, but I will say that experiences like the integrated HTML documentation currently provided with Torque X and TGB 1.5.1, the tutorial pack that was recently released for Torque X, and similar efforts are just scratching the surface of what the productization of Torque 2 will be all about.
#31
10/18/2007 (4:36 pm)
Quote:
@Mark Dynna
I support your pleas here. I can only imaging there will be at least one person who will not agree with what is going. I hope that the communities/supporters of GG maintains the higher more mature standard. In other words ignore those who don't have valid comments. If you have something negative to say just make sure you support and document it with good sense. :)

I will promise you this: the community participation standards and requirements in the Transparent Development Community will be much higher than they are in our current product community. We absolutely must have bi-directional communication, including strong debate around all sides of any particular issue, but I will be enforcing 100% professionalism and mutual respect at all times.

I wish the infrastructure was in place myself right now so we could get this party started--but I have strict requirements for how that community can be managed, and it requires a lot of changes to how our web infrastructure works, so it will take time for it to be implemented. The current GG.com community is a mixed one, with many purposes and levels of interaction. The Transparent Development Community will have one purpose: provide a community focused on the development of Torque 2, and I will be actively moderating at all times to make sure it stays on focus.

The current GG.com community will always exist, and general discussions, questions unrelated to Torque 2 development, soapboxes, rants, product feedback (as opposed to Torque 2 development feedback) and other off topic/unrelated discussion is going to simply not be allowed in the TDC.
#32
10/18/2007 (4:37 pm)
I too support a refactoring of Torque to a component-based architecture. I think this will be the window through which many unresolved problems might finally be fixed. My one gripe would be the amount of money I've spent on licenses and soon to be outdated resources. Is there a chance current license owners can get grandfathered in or at least receive a discount?

With regard to T2 polysoup will it have lighting similiar to BSP so interiors can built with it?
#33
10/18/2007 (4:45 pm)
The "Grandfathering Issue" (as well as the related "existing product sunset" issue) is one that has been on our minds for the last 4 months. We don't have a 100% solution yet to the grandfathering issue, so I'm not going to set and false expectations, but it continues to be of extremely high importance in our minds, and right now no solution is off the table, from the minimal impact (give discounts) to the completely radical (???).

This particular debate will continue to be held internally only--I'm willing to listen to various ideas (elsewhere please, not in this thread), but we will most likely not publicly respond until we have a solution that not only makes us happy, but will make you happy as well.

Quote:
With regard to T2 polysoup will it have lighting similiar to BSP so interiors can built with it?

Lighting is totally abstracted--if you add a lighting component to your object, it will receive whatever lighting you wish (and have available), including third party solutions. That includes pre-generation of static lightmaps, dynamic lighting, or whichever modules you have included, and component properties you have selected.
#34
10/18/2007 (5:33 pm)
Might I suggest that as long as you are going with a "constant compile" model that all persons on the development team get the same source formatter software, and that check-in cannot be done until the source formatter has been run?

You can have one big argument at the beginning as to what the settings are for the source formatter and then everybody has to live with the results.

Looking way down the road you might even tell us which formatter you chose and give us your settings so that we can make suggestions for improvements - properly formatted for inclusion.

Just a thought. I have found that it does save a remarkable amount of time when everybody's work looks the same.
#35
10/18/2007 (5:38 pm)
Aww man, just when I was getting the hang of things with the existing product line!

Well, bring it on, I say.
#36
10/18/2007 (6:12 pm)
Another question regarding the TNL networking component:

Will the OpenTNL be included or the "older" TNL network layer from TGE? The reason I ask is that afaik OpenTNL has some major advances like firewall punch-through and secured sessions compared to the "older" implementation in TGE.
#37
10/18/2007 (6:17 pm)
Little confusion of terms I guess--TNL and Torque Networking are two different things, so when I say TNL, I mean the "new" stuff, not what is existing in the current engines. TNL is of course an "upgrade" of Torque Networking in some ways, as you mentioned.
#38
10/18/2007 (6:18 pm)
Ahhhh, yes yes yes yeeessssss! That's bloody fantastic news! Thanks again for this news about TNL Stephen. *MartinHappy* :-)
#39
10/18/2007 (6:26 pm)
Thanks for reply Stephen.
The localization / multi-language support is important for us (non-English developers). No, we don't plan to "ship a game within a year", but we are working on MMO, and it's a question of constant development/improvements.
One MMO game is moving from Gamebryo into CryEngine2, so why can't we go from T1 to T2 then? :)

I'll try pay lots of attention on the T2 development, hope it will go nicely. Torque - forever! :)

Cheers!
#40
10/18/2007 (6:36 pm)
Very important question which I think we are all wondering....

Will Kork be the demo character in T2?

:)