Transparent Development, Torque 2, and You!
by Stephen Zepp · 10/17/2007 (10:26 pm) · 122 comments
There has been some buzz floating around since IGC this year, and a lot of it has been around InstantAction, but we also announced a couple of additional topics that are starting to sink in, so I wanted to dive in to the deep end of the community pool and make some waves....
Transparent Development
Sounds nifty, but what is it? One analogy I've heard that is a pretty solid description is consider TD a window into the offices of the development staff working on Torque 2--see what's being worked on this week, read discussions about designs as they are being discussed internally, and most importantly be able to discuss the designs and implementations themselves, as they occur.
We're not talking marketing information for a product that has already been developed--we're talking watching, and even being able to discuss and provide feedback on the development of Torque 2 as it occurs. I'll get to Torque 2 in a moment, but I want to reinforce that this is a fundamental shift back to the way GarageGames used to interact with the community--away from keeping completely silent regarding what we're working on and why, and back to giving the community rapid and fully packed pages of information on the development as it occurs--not after the fact, and especially not after it's released. Torque 2 isn't even scheduled for beta until well in to 2008, but we're working as rapidly as we can to get infrastructure in place to set up discussions as the engine team works on development.
We're talking roadmaps, milestone lists, design write ups, expected work flow and implementation standards, the works--all up front, all the time. One of the projects that I am working on right now is designing a new set of forums, wiki, and related web tools to manage extensive and extended data flow between the Transparent Development Community (that's you guys) and the engine development team (that's us!). Once this infrastructure is in place, we're going to open it up for communication purposes to a limited set of users for beta testing for a few weeks, and then we will make the information available to all.
[edit: see below in the comments section for further clarification on content and timing]
It's going to be several weeks yet before this infrastructure is ready, and it will be a few weeks of beta testing of that infrastructure as well before we're ready to turn on the fire hose, so please be patient while we get things set up for this to work in the long term--when it's ready, we'll let everyone know and let the show begin!
I do want to make something fundamentally clear from the beginning (and trust me, I'll keep pressing this point until it sinks in fully!): Torque 2 development will be an iterative process, very similar to the SCRUM, or Agile/Spiral Development models--and one of the key aspects of this style of development is that sometimes you go down a path, and it turns out that the path is not the best solution. When this occurs, the best course of action is simply to cut out the results of that development spiral, and reset your project expectations. Sometimes this means removing features, adjusting the milestone list, or delaying (or in some cases even canceling) entire systems.
It is critical that the entire Transparent Development Community understand that this development process you will be watching and discussing is a fluid one--and it should be fully expected that features and systems being discussed one week may very well end up being cut forever the next. No projects, games, and especially no funds or contracts should ever be committed during this development process--if you need to make commitments or financial decisions regarding your projects, then the Torque 2 Transparent Development Community is absolutely not somewhere you should be.
Torque 2
So what's Torque 2? To put it simply, Torque 2 is a ground up re-factor of the entire architecture of all of our engines into a new engine with some pretty amazing architecture and features. We've learned quite a bit over the last several years of engine development, and the time has come to take those lessons and build a new engine that not only provides the maximum flexibility possible to game developers, but fundamentally change many of the designs of the engine itself--correcting design assumptions and work flows that came about over the years that we ourselves find too restrictive, and along the way providing customization and feature enhancement capabilities that were never available before.
Torque 2's fundamental changes fall under three primary areas:
Modularity of Systems: In the past, it was very difficult to leverage into Torque systems, libraries, and raw code written by 3rd parties, such as ODE/Havoc/PhysX Physics, alternate scripting languages such as Lua, Python, and even .NET languages, and many other cool things that are available to game developers. Torque is extremely integrated, with each of the systems tightly coupled to provide a powerful, but ultimately restrictive engine when it comes to "mashing up" other solutions into a coherent whole--and Torque 2 is designed from scratch to correct those issues by allowing game developers to replace major system modules in a "plug and play" style--as long as the external systems match interfaces that the Torque 2 framework expects, you will be able to use just about any 3rd party (or self-developed) system within the core Torque 2 framework.
Abstraction and Interfaces: One of the key ways we will be able to provide this type of "swap out" modularity is by abstracting most of the core systems via the use of interfaces. Each major system will describe a set of interfaces that other systems and objects will use to communicate with the system, so instead of calling code directly within a system, external modules and objects will interact with the interfaces that system exposes--allowing other systems to replace core modules at the whim of the developer.
Torque 2 has already leveraged the previously developed GFX abstraction layer present in TGE-A, and in addition has had several iterations of abstraction of core systems as well, such as the sound layer (now abstracted through a new SFX system), lighting (fully abstracted into a separate module), collision and physics (currently under iterative refactoring towards a comprehensive interface), and many others. As the iterative development spirals of Torque 2 continue, additional systems will be abstracted and interfaces implemented for even more comprehensive modularity.
Components: The concept of a component based architecture is actually very simple--but it has some far reaching design implications, and in many cases can take some time to adapt towards. In most game engines, including Torque, you have a very deep set of classes that form a pretty static ancestry, and you use inheritance of all of these classes in your game object level classes. Unfortunately, this has several issues that become more apparent the longer you use these classes for development, especially across multiple games: your game object classes start becoming more and more bloated, and less appropriate for specific uses. Anyone that takes a good look at the Torque ShapeBase class will soon see what I mean--there are still left over methods in the ShapeBase class that are part of original Tribes code, and aren't particularly useful for most games.
Here's an example of exactly what I mean: Back when BraveTree was working on their environment pack, one of the neat things they wanted to have was butterflies with random movement about the scene. Pretty nifty stuff--they derived a new class off of ShapeBase, and their butterflies merrily wandered about the stronghold mission randomly--until one of them happened to run in to a crossbow object placed in the scene.
Suddenly, this tiny little butterfly picks up this huge crossbow, and starts swinging it back and forth madly as it continues to wander about--merrily meandering about the scene with a totally unexpected weapon of mass destruction dangling from its cute little mandibles.
It was immediately obvious what was going on, and as soon as they picked themselves up from off the floor and stopped laughing, they realized that since the butterfly inherited from ShapeBase, it also inherited the ability to collide with Items, and mount weapons--and that's exactly what it did!
So what does this have to do with components? Well, with the idea of components (based on the Object Oriented Programming concept of aggregation, sometimes called composition), instead of inheriting a series of ever more complex functionalities that your particular object may not need, you create very small units of functionality that accomplish simple tasks--and "build" your game objects from these little components. In the case of the butterfly, they might have added a render3D component, a collision3D component, a hoverFlightMovement component, and a AIRandomMove3D component--and thereby had all the functionality they needed for the butterfly, without any of the bloat that inheriting from a full class would have given them.
We'll be covering components in quite a bit more depth once the Transparent Development Community is in place, but if you are interested in learning more about how component based game development works, TorqueX also uses components, and is a great place to learn how the new design concepts can be used.
What this means for You
Well, the very first thing it means is not a lot to be honest--this is an announcement of a new development style (transparency), and a new engine that is just now entering heavy development. Torque 2 is not going to be ready for use in a major (or even minor) game project until sometime in 2008 (more accurate time line will come when we get more iteration cycles finished up). If you are currently working on a game with Torque (or another engine), or are planning on starting a new project in the next 6 to 9 months, Torque 2 is not for you at this time.
If you are someone that is interested in how an engine is developed, and want to participate in discussions of that development process as it occurs, stay tuned for further announcements about the Transparent Development Community. Once it opens to the public stop on by and hang out--at the least it should be educational and informative, and hopefully some of you might even generate ideas, designs, and implementations that may very well be factored in to the future of Torque.
I'm already well over my word count for a .blog, so I'll answer specific questions in the comment area below.
Transparent Development
Sounds nifty, but what is it? One analogy I've heard that is a pretty solid description is consider TD a window into the offices of the development staff working on Torque 2--see what's being worked on this week, read discussions about designs as they are being discussed internally, and most importantly be able to discuss the designs and implementations themselves, as they occur.
We're not talking marketing information for a product that has already been developed--we're talking watching, and even being able to discuss and provide feedback on the development of Torque 2 as it occurs. I'll get to Torque 2 in a moment, but I want to reinforce that this is a fundamental shift back to the way GarageGames used to interact with the community--away from keeping completely silent regarding what we're working on and why, and back to giving the community rapid and fully packed pages of information on the development as it occurs--not after the fact, and especially not after it's released. Torque 2 isn't even scheduled for beta until well in to 2008, but we're working as rapidly as we can to get infrastructure in place to set up discussions as the engine team works on development.
We're talking roadmaps, milestone lists, design write ups, expected work flow and implementation standards, the works--all up front, all the time. One of the projects that I am working on right now is designing a new set of forums, wiki, and related web tools to manage extensive and extended data flow between the Transparent Development Community (that's you guys) and the engine development team (that's us!). Once this infrastructure is in place, we're going to open it up for communication purposes to a limited set of users for beta testing for a few weeks, and then we will make the information available to all.
[edit: see below in the comments section for further clarification on content and timing]
It's going to be several weeks yet before this infrastructure is ready, and it will be a few weeks of beta testing of that infrastructure as well before we're ready to turn on the fire hose, so please be patient while we get things set up for this to work in the long term--when it's ready, we'll let everyone know and let the show begin!
I do want to make something fundamentally clear from the beginning (and trust me, I'll keep pressing this point until it sinks in fully!): Torque 2 development will be an iterative process, very similar to the SCRUM, or Agile/Spiral Development models--and one of the key aspects of this style of development is that sometimes you go down a path, and it turns out that the path is not the best solution. When this occurs, the best course of action is simply to cut out the results of that development spiral, and reset your project expectations. Sometimes this means removing features, adjusting the milestone list, or delaying (or in some cases even canceling) entire systems.It is critical that the entire Transparent Development Community understand that this development process you will be watching and discussing is a fluid one--and it should be fully expected that features and systems being discussed one week may very well end up being cut forever the next. No projects, games, and especially no funds or contracts should ever be committed during this development process--if you need to make commitments or financial decisions regarding your projects, then the Torque 2 Transparent Development Community is absolutely not somewhere you should be.
Torque 2
So what's Torque 2? To put it simply, Torque 2 is a ground up re-factor of the entire architecture of all of our engines into a new engine with some pretty amazing architecture and features. We've learned quite a bit over the last several years of engine development, and the time has come to take those lessons and build a new engine that not only provides the maximum flexibility possible to game developers, but fundamentally change many of the designs of the engine itself--correcting design assumptions and work flows that came about over the years that we ourselves find too restrictive, and along the way providing customization and feature enhancement capabilities that were never available before.
Torque 2's fundamental changes fall under three primary areas:Modularity of Systems: In the past, it was very difficult to leverage into Torque systems, libraries, and raw code written by 3rd parties, such as ODE/Havoc/PhysX Physics, alternate scripting languages such as Lua, Python, and even .NET languages, and many other cool things that are available to game developers. Torque is extremely integrated, with each of the systems tightly coupled to provide a powerful, but ultimately restrictive engine when it comes to "mashing up" other solutions into a coherent whole--and Torque 2 is designed from scratch to correct those issues by allowing game developers to replace major system modules in a "plug and play" style--as long as the external systems match interfaces that the Torque 2 framework expects, you will be able to use just about any 3rd party (or self-developed) system within the core Torque 2 framework.
Abstraction and Interfaces: One of the key ways we will be able to provide this type of "swap out" modularity is by abstracting most of the core systems via the use of interfaces. Each major system will describe a set of interfaces that other systems and objects will use to communicate with the system, so instead of calling code directly within a system, external modules and objects will interact with the interfaces that system exposes--allowing other systems to replace core modules at the whim of the developer.
Torque 2 has already leveraged the previously developed GFX abstraction layer present in TGE-A, and in addition has had several iterations of abstraction of core systems as well, such as the sound layer (now abstracted through a new SFX system), lighting (fully abstracted into a separate module), collision and physics (currently under iterative refactoring towards a comprehensive interface), and many others. As the iterative development spirals of Torque 2 continue, additional systems will be abstracted and interfaces implemented for even more comprehensive modularity.
Components: The concept of a component based architecture is actually very simple--but it has some far reaching design implications, and in many cases can take some time to adapt towards. In most game engines, including Torque, you have a very deep set of classes that form a pretty static ancestry, and you use inheritance of all of these classes in your game object level classes. Unfortunately, this has several issues that become more apparent the longer you use these classes for development, especially across multiple games: your game object classes start becoming more and more bloated, and less appropriate for specific uses. Anyone that takes a good look at the Torque ShapeBase class will soon see what I mean--there are still left over methods in the ShapeBase class that are part of original Tribes code, and aren't particularly useful for most games.Here's an example of exactly what I mean: Back when BraveTree was working on their environment pack, one of the neat things they wanted to have was butterflies with random movement about the scene. Pretty nifty stuff--they derived a new class off of ShapeBase, and their butterflies merrily wandered about the stronghold mission randomly--until one of them happened to run in to a crossbow object placed in the scene.Suddenly, this tiny little butterfly picks up this huge crossbow, and starts swinging it back and forth madly as it continues to wander about--merrily meandering about the scene with a totally unexpected weapon of mass destruction dangling from its cute little mandibles.It was immediately obvious what was going on, and as soon as they picked themselves up from off the floor and stopped laughing, they realized that since the butterfly inherited from ShapeBase, it also inherited the ability to collide with Items, and mount weapons--and that's exactly what it did!
So what does this have to do with components? Well, with the idea of components (based on the Object Oriented Programming concept of aggregation, sometimes called composition), instead of inheriting a series of ever more complex functionalities that your particular object may not need, you create very small units of functionality that accomplish simple tasks--and "build" your game objects from these little components. In the case of the butterfly, they might have added a render3D component, a collision3D component, a hoverFlightMovement component, and a AIRandomMove3D component--and thereby had all the functionality they needed for the butterfly, without any of the bloat that inheriting from a full class would have given them.
We'll be covering components in quite a bit more depth once the Transparent Development Community is in place, but if you are interested in learning more about how component based game development works, TorqueX also uses components, and is a great place to learn how the new design concepts can be used.
What this means for You
Well, the very first thing it means is not a lot to be honest--this is an announcement of a new development style (transparency), and a new engine that is just now entering heavy development. Torque 2 is not going to be ready for use in a major (or even minor) game project until sometime in 2008 (more accurate time line will come when we get more iteration cycles finished up). If you are currently working on a game with Torque (or another engine), or are planning on starting a new project in the next 6 to 9 months, Torque 2 is not for you at this time.
If you are someone that is interested in how an engine is developed, and want to participate in discussions of that development process as it occurs, stay tuned for further announcements about the Transparent Development Community. Once it opens to the public stop on by and hang out--at the least it should be educational and informative, and hopefully some of you might even generate ideas, designs, and implementations that may very well be factored in to the future of Torque.
I'm already well over my word count for a .blog, so I'll answer specific questions in the comment area below.
#102
For anyone used to OOP it is a thousand times quicker to understand good interface docs than to trawl through long-winded explanations on TDN or in books. Of course those long explanations are essential when you don't understand OOP (or art, which I don't) or whatever! But when you are buried deep in code and want to know quickly "Does this interface have a getString method" then tech docs are the quickest way to find out without losing your concentration.
11/14/2007 (2:37 am)
Back to the development discussion - Can GG make sure that a good set of UML diagrams are available for all of the interfaces please (when they are stable of course). Also produce some really good quick-ref tech docs on the interfaces (a la Javadocs). If you choose your code documentation tool before you start, all of your code comments can be in the correct format to auto-produce the docs when ready. For anyone used to OOP it is a thousand times quicker to understand good interface docs than to trawl through long-winded explanations on TDN or in books. Of course those long explanations are essential when you don't understand OOP (or art, which I don't) or whatever! But when you are buried deep in code and want to know quickly "Does this interface have a getString method" then tech docs are the quickest way to find out without losing your concentration.
#103
JoZ
11/22/2007 (5:02 am)
What I wanna ask is if in the meantime GG will continue to develop TGEA till Torque 2 isn't comes out, since it continue to lack of some important features announced time before...JoZ
#104
There might be a few fixes here and here, but you should consider TGEA as a frozen engine.
Which means Open GL support is dead too. 8-(
I have one question to Stephen: what will happen to TGE and TGEA once TGE2 will be released?
Are they going to be available for free or with very very low amount of money to encourage hobbyists and apprentice programmers to use them?
11/22/2007 (5:10 am)
@Joz: no.There might be a few fixes here and here, but you should consider TGEA as a frozen engine.
Which means Open GL support is dead too. 8-(
I have one question to Stephen: what will happen to TGE and TGEA once TGE2 will be released?
Are they going to be available for free or with very very low amount of money to encourage hobbyists and apprentice programmers to use them?
#105
11/22/2007 (5:11 am)
Hmm, that means that the money I invested for TGEA seems dead money now. Very sad.
#106
11/22/2007 (5:13 am)
I think they will provide a good switching plan for TGEA owners. It would be fair. 
#107
11/22/2007 (8:43 am)
depends. I would say if your starting your project in the next say year and a half or so then you should just use TGEA. I mean I think it gonna be a while before here is anything that is EA or even Beta or what ever that we are gonna be able to get a hold or I don't think. I would say year to maybe 2 beore it is anything you are gonna release a title with. I mean realisticly.
#108
So currently, GG is not providing anything for that purpose but TGE, and many may switch to different engines, C4, Unity, etc.
11/22/2007 (8:49 am)
@James: it is simply bad if you were looking to use TGEA as a cross-paltform development.So currently, GG is not providing anything for that purpose but TGE, and many may switch to different engines, C4, Unity, etc.
#109
They said 2008 and they will likely make it in time with their new resources from IAC.
Juggernaut currently already exist, it simply needs to be finished into Torque 2.
11/22/2007 (8:56 am)
@JamesThey said 2008 and they will likely make it in time with their new resources from IAC.
Juggernaut currently already exist, it simply needs to be finished into Torque 2.
#110
11/22/2007 (10:25 am)
@Stephan: Absolutly right. I'm already owner of C4 and already playing with Unity (hot candidate for me!).
#111
11/22/2007 (12:13 pm)
I think this me too... money spent in TGEA has not been very full of value but certainly they will provide a good discount for TGEA owners... and the new product for sure will be far better from the base to up... :)
#112
--TGE-A (and our other products) is not "frozen". I was just in a meeting yesterday discussing options and possibilities for our various existing engines. As mentioned, Torque 2 is quite a ways out, and we continue to assess and plan for existing engines.
--If it helps you to set up the expectations properly for Torque 2, it should be considered "early research and development"--we're just letting you know about it (and in the future, being able to watch the process) much earlier than you are used to.
--We've discussed this many times in many forums, posts, and other discussion areas, but features that have in the past been described as possibilities (or even goals) sometimes simply don't happen--and this means that there will always be certain people or groups that will say "I bought XXX for feature YYY, and it didn't get developed!". I'm not going to try to convince each and every single person that the value of TGE-A compared to the money they spent is a huge value in just about any economic development model, but as with any game project decision, if you find the path that you've stepped on isn't working for your team's needs, then it's important to correct that decision in whatever way is appropriate.
--We have spent quite a bit of time researching and planning for various transition strategies to Torque 2, and once it makes sense to discuss them publicly, we will announce how, why, and "how much" the transition for existing customers will be. It will be several months in the future before Torque 2 will be ready to even begin considering this scenario as an end user, so right now I'd suggest you not even think about it :)
11/22/2007 (12:35 pm)
Lots of speculation going on recently, and while I can't give many details about specifics (my area is Transparent Development of Torque 2, not other engines or products), I did want to correct a few inconsistencies.--TGE-A (and our other products) is not "frozen". I was just in a meeting yesterday discussing options and possibilities for our various existing engines. As mentioned, Torque 2 is quite a ways out, and we continue to assess and plan for existing engines.
--If it helps you to set up the expectations properly for Torque 2, it should be considered "early research and development"--we're just letting you know about it (and in the future, being able to watch the process) much earlier than you are used to.
--We've discussed this many times in many forums, posts, and other discussion areas, but features that have in the past been described as possibilities (or even goals) sometimes simply don't happen--and this means that there will always be certain people or groups that will say "I bought XXX for feature YYY, and it didn't get developed!". I'm not going to try to convince each and every single person that the value of TGE-A compared to the money they spent is a huge value in just about any economic development model, but as with any game project decision, if you find the path that you've stepped on isn't working for your team's needs, then it's important to correct that decision in whatever way is appropriate.
--We have spent quite a bit of time researching and planning for various transition strategies to Torque 2, and once it makes sense to discuss them publicly, we will announce how, why, and "how much" the transition for existing customers will be. It will be several months in the future before Torque 2 will be ready to even begin considering this scenario as an end user, so right now I'd suggest you not even think about it :)
#113
yeah end of 2008 at best I would say. and even then I would not say that it is going to be in a condition to produce a commercial title.
@Stephan:
dude you really intend that? I don't think anyone realisticly actually expects OpenGL support anymore for some time now. And as an indie do you really have the funds to build and test and support a linux or openGL version as well for so few users? I have said it many times trying for all the platforms is just going to spread an indie too thin.
Anyway that is my opinion. Personaly I say make an engine that is a DirectX 10 capable power house. Something capable of viually supassing Crysis should be the goal. That game is the State of the art today and No one is going to even be able to condsider releasing a title with torque 2 for prolly 2 year minimum. I hope Torque 2 is something that does not look like it is a few years old by the time it is released.
11/22/2007 (8:07 pm)
@ ben:yeah end of 2008 at best I would say. and even then I would not say that it is going to be in a condition to produce a commercial title.
@Stephan:
dude you really intend that? I don't think anyone realisticly actually expects OpenGL support anymore for some time now. And as an indie do you really have the funds to build and test and support a linux or openGL version as well for so few users? I have said it many times trying for all the platforms is just going to spread an indie too thin.
Anyway that is my opinion. Personaly I say make an engine that is a DirectX 10 capable power house. Something capable of viually supassing Crysis should be the goal. That game is the State of the art today and No one is going to even be able to condsider releasing a title with torque 2 for prolly 2 year minimum. I hope Torque 2 is something that does not look like it is a few years old by the time it is released.
#114
I think the IAC deal kinda changed that ;)
11/22/2007 (8:27 pm)
Quote:And as an indie do you really have the funds...?
I think the IAC deal kinda changed that ;)
#115
11/23/2007 (6:53 am)
Quote:I don't think anyone realisticly actually expects OpenGL support anymore for some time now.I expect it.
#116
@tom
How does that change that about of money you have to spend on development? Sure GG has more money to spend but we are developers do not.
11/23/2007 (7:35 pm)
will if you notice it doesn't say anything about OpenGL on the Features page anymore.@tom
Quote:I think the IAC deal kinda changed that
How does that change that about of money you have to spend on development? Sure GG has more money to spend but we are developers do not.
#117
Ahh, I see now that I misinterpreted the 'you' in your post. I thought you were speaking to Stephen/GG and not about indies in general. That being the case, my point was worthless, though I still think cross-platform support is a key point for indies nevertheless. Even without resources to focus on all platforms, the possibility should exist to do so or to choose a non-mainstream one.
11/23/2007 (7:46 pm)
@JamesAhh, I see now that I misinterpreted the 'you' in your post. I thought you were speaking to Stephen/GG and not about indies in general. That being the case, my point was worthless, though I still think cross-platform support is a key point for indies nevertheless. Even without resources to focus on all platforms, the possibility should exist to do so or to choose a non-mainstream one.
#118
If you are correct and T2 is released without OpenGL support, I will reassess my options.
Edit reason: Grammar
11/24/2007 (7:19 am)
Quote:will if you notice it doesn't say anything about OpenGL on the Features page anymore.I'm giving GG the benefit of the doubt on this one. They said that there would be OpenGL support at IGC and I'm going to wait and see it actually happens.
If you are correct and T2 is released without OpenGL support, I will reassess my options.
Edit reason: Grammar
#119
when will you come with more information?
11/30/2007 (3:09 pm)
I hope this engine will not release sometime in winter 2008... and I hope for som screenshots to ;) when will you come with more information?
#120
11/30/2007 (4:03 pm)
Why do you want it to not release in the winter of 08?
Torque Owner Albert Hammel
"All of the hating is not going to help except to make the people at GG bitter and go back to no communicating with us."
When you are dishonest with your customers, they typically leave. This is something GG needs to start getting right or they will suffer the fate of lots of other companie that treat their customers like feces. Why are you worried about them becoming secretive? It seems they have finally learned their lesson. You need to cater to your market. I'm amazed that they have held on this long.