Why WAS Torque3D documentation so poor?

#341 And I thought I was the angry one here. bring it down a few notches, eb. Insults get you nowhere.

Discalimer for the following: Informative, not angry or accusatory.

@ Eric - Thanks for the tangent. Sometimes it is easy for people who are new to the industry or who have gotten used to the new way of game-making to lose sight of the fact that this really is how things were done back in the day (read: a few years ago). I am in that category.

This doesn't mean that we're going to be any more forgiving, however. The community and industry have changed, and it was a hard, rapid change. It was easy for Unity to adapt because they are one of the main reasons for the change, and with the manpower and talent base that Epic Games has, it was just as easy for them to shrug and crank out a freebie engine and docs out the ying-yang.

It seems that GG was distracted with its massive product line and the release of a new flagship and got caught unawares. Just because I'm currently a student doesn't mean I'm young, or oblivious to the nuances of running a business. This industry doesn't allow for many mistakes like that. A company constantly has to have an ear to the ground and has to be able to adapt quickly.

The game design industry as a whole has changed. Things went from literally designing games in your own garage to mega-corporate schemes, and now it's see-sawing back to indie developers. The companies positioning to make money off of this wave have to be flexible, they have to be ready, and they have to respond to the customers as quickly as possible.

As you no doubt have noticed, we (customers as a whole) are a finicky, easy-to-anger, unpredictable, and easy-to-lose bunch these days.

#342 Eric, this isgreat to read. I am one of these hobby devs hoping to one time finish a game, and the best that can happen to me is that T3D gets "easier" to approach. It would be great if more could be achieved in script....if script access would go so far as it goes in unity...then I think T3D would be even more valuable and together with good docs, it would kick the others.
What makes T3D difficult for people that are not good at c++ is that there are things which should be doable in script, but can only be done in C++, like for example fully customizing the camera.

as pino already said, great thread and even better than it gets not deleted here...I really have the feeling that GG is listening to the community and more and more I believe GG is heading in the right direction. Can't wait for 1.1!

#343 UPDATE:

Alright, I just hit the "GO" button. An updated version of both the web docs (docs.torquepowered.com/torque-3d/official) and the TorqueScript Manual (CHM). Are uploading. They should be viewable as soon as the web cache is flushed, which takes about 45 min.

NOTE: These are not the finalized docs, just a big update. Please post any feedback, errors discovered, and critical issues with this update in the Official Documentation Feedback Thread.

#344 It's good to be excited.

I can tell you that internally we are excited too, but there is a lot of work to do. I personally have taught over 1000 game development students about engines while at Full Sail so I think I have a reasonable understanding of the level that it takes to bring casual and beginner users up to speed on complex topics. We've done a lot of work to get the ball rolling and from my internal perspective, it's all about execution.

@Tom - keep holding us to a high standard...it's important to us and helps us set expectations appropriately. I think it's important for us to realize, with resource constraints and a lot of products ( something we are fixing as well ), that we can't instantly deliver everything we would like to as fast as we would like to. But what we can do is set expectations on what we can do honestly and meet or exceed those expectations. Understanding your expectations is the first part in this process.

#345 Eloquent and well said Giuseppe.

I dont think i would be a Torque community member if they censored realistic criticism and controlled what opinions or debates were allowed .

I love the fact that the TorquePowered officials will participate in such debates, it is an important defining action that should prove to the community their concerns and opinions do matter and are being considered.

It is very rare to be seen a Torque community member who is simply ranting without comprehension or trying to pick fights on a personal level against opposing opinions. (Example:eb's comment seems logically sound. Everyone should exercise right to keep there statements clarified.)

Now off to explore the newly updated Docs.

#346 I'm in IRC Hour right now answering questions about the docs and other Torque news, just FYI

#347 Where are the new docs?

#348 Official Docs

Script Manual

Note: This is the permanent link to the CHM. All future updates will be found at that link.

#349 Michael Perry working late, replying to forum posts long after work hours, on a Friday night, in the middle of this hot summer.

If no one else see that as a clear definition of TorquePowers desire to get everything made right, then perhaps ranting without comprehension Torque community members are not as rare as i had always wanted to believe.

#350 It's Friday? O_o

I'm starting to lose sense of time

#351 Thanks Mich. I've just downloaded the chm. Looking forward to seeing a bit more in it. I am also looking forward to more on this. I'm also awaiting the next T3D blog/announcement, and the demos. Demos can wait but I am looking forward to 1.1 just because of a lot of the fixes. My co-worker is really awaiting our work with Torque as am I.

edit: @Caylo: That's the spirit!
@Mich and Eric: Keep it up guys!

[below offtopic]

@Gerry: Those might have been directed at me. Not sure, but meh...

@Eric: Either way I'm looking forward to future releases. I've learned a lot. So far I think everything may be going well. Hopefully you didn't read my post here and think bad of it.

Quote:As of yet they have not lied about the new release, and I don't think they will.

That was meant to the defense of GG.

Quote:Anyways, my point is...back in the day we catered to programmers and that was fine because almost everyone making games either was a programmer or relied on one. Times have changed.

I don't say this out of anger or anything like that. This is just to answer to it.

I said I didn't want to have to go through source, although if I plan to do certain things I probably am going to have to go through it either way. Mostly the reason I want docs is to get a reference guide. I'm not looking for tutorials, just API. Also, not having documentation for at least that is alienating your audience. TBH if Torque had good docs and tutorals when I was a complete noob, I might have considered it instead of 3DGS. Me and my co-worker decided on Torque now because it had moved along really well and looks better than it did before. From the time we got that we have released two games, thank God, and have learned lots from day one to now, about 5 years later.

I mentioned that to say this: Better Docs = More Customers, especially those new to game development. Of course a lot of them are going to ask "how do I make a [insert game-type here]?". There will be lots of new people asking how to do everything, that's how it goes (and that's how we all started), but essentially it comes down to documentation being an extra that's convenient to have for some with at least some experience. Docs are not required to make a game, but it does help you create a game on an engine that you are new to without taking more time than needed to learn what the very basics are. I'm looking at a possible 2 year development on our next title, and that's if we have good documentation and catch on quickly with our new toolset (which I really really want to get into).

Overall, look at it this way. Torque is a breath of fresh air for me. 3DGS was...good to learn on (and without it MangaPage wouldn't have released 2 games yet), but the tools were...not very friendly at times. I am looking forward to Torque a lot, and so far I think you guys have been ok. I look forward to more news.

#352 Reminder: you may need to right click, go to properties, and click 'unblock' to view the file.

Nice torquescript reference. this oughta shut people up for awhile =)

only 3 suggestions:

1. Indicate on the objects description pages whether they can be instanced directly. Gamebase and shapebase objects cannot be instanced through script.

2. Descriptions of objects derived from shapebase should include references to the callbacks they inherit.

3. The inheritance diagrams should be expanded by default.

#353 @Sean - Excellent suggestions. I'll see what we can do to implement them.

#354 NOTE: THE FOLLOWING FOUR OR FIVE POSTS ARE FOR THOSE INTERESTED IN HOW THE NEW DOCUMENTATION PROCESS HAS BEEN MADE POSSIBLE. ALSO VERY USEFUL FOR THE C++ PROGRAMMERS WHO WANT A HEADS UP ON WHAT'S CHANGED IN THE ENGINE:

By the way, I feel like providing a little insight on what's going on underneath the hood. What most of you have been seeing is the final product of the work we have been putting into the docs. What you will not see until Beta 2 is what has changed in the engine to make this possible. I'll preface all of this with my sympathy to anyone who is going to be performing manual merges, because 95% of the documentation is a result of writing strings inside of C++ and exporting via a Doxygen system. So when you see this flood of docs, it means another source file has changed.

With that said, how is this being accomplished? Well, if you have ever dug into the source you probably already know how functions, methods, and variables are exposed. This is through our console system, using various macros, reflection standards, and C++ hooks. We are changing a lot and using new macros to accomplish this automated documentation process. The correlation is pretty simple:

Global functions -> ConsoleFunction(...)
Class methods -> ConsoleMethod(...)
Member variables -> addField(...)
Callbacks -> Con::executef(...)

We are changing the macros we use to fill in the undocumented sections of the script API:

Global functions -> DefineEngineFunction(...)
Class methods -> DefineEngineMethod(...)
Member variables -> addField(...) mostly the same
Callbacks -> Something completely new, described later

Let's break down what has changed...

Global functions: ConsoleFunction(...)
These are stand alone functions that can be called in TorqueScript without belonging to any class or object.

In TorqueScript, it is called this way:

echo("Hello World");

In C++ they were defined by the following:

ConsoleFunction(addBadWord, bool, 2, 2, "addBadWord(someReallyNastyWord);")
{
   TORQUE_UNUSED(argc);
   return gBadWordFilter->addBadWord(argv[1]);
}

What you see as the usage string ("words here"), was able to be dumped to the console in the old days. Wow...what sucky documentation. We have changed that.

The new macro we use allows for much more robust documentation:

DefineEngineFunction(addBadWord, bool, (const char* badWord),,
					 "Add a string to the bad word filtern"
					 "The bad word filter is a table containing words which will not be "
					 "displayed in chat windows. Instead, a designated replacement string will be displayed.n"
					 "@param badWord Exact text of the word to restrict.n"
					 "@return True if word was successfully added, false if the word or a subset of it already exists in the tablen"
					 "@see filterStringnn"
					 "@tsexamplen"
						"// In this game, "Foobar" is bannedn"
						"%badWord = "Foobar";nn"
						"// Returns true, word was successfully addedn"
						"addBadWord(%badWord);nn"
						"// Returns false, word has already been addedn"
						"addBadWord("Foobar");"
					 "@endtsexamplen"
					 "@ingroup Game")
{
	TORQUE_UNUSED(badWord);
	return gBadWordFilter->addBadWord(badWord);
}

So if you open up the new TorqueScript Reference Manual chm, you can do a search for addBadWord, or check it out under the Modules->Game header.

We can document parameters, descriptions, and use examples. Very nifty, very useful, very happy this is automatically dumped to a more human readable format. Check it out in the new CHM.

(continued in the next post for ConsoleMethods)

#355 Class methods: ConsoleMethod(...)
These are functions called in TorqueScript from an object associated with a class, such as SFXSource.

In TorqueScript, we call them this way:

%sfxSourceObject.stop()

Here is the old macro for defining a ConsoleMethod:

ConsoleMethod( SFXSource, stop, void, 2, 3,  "( bool immediate=false ) - Ends playback of the source." )
{
   bool immediate = false;
   if( argc > 2 )
      immediate = dAtob( argv[ 2 ] );
      
   object->stop( immediate );
}

And here is the new way:

DefineEngineMethod( SFXSource, stop, void, ( F32 fadeOutTime ), ( -1.f ),
   "Stop playback of the source.n"
   "@param fadeOutTime Seconds for the sound to fade down to zero volume.  If -1, the SFXDescription::fadeOutTime "
      "set in the source's associated description is used.  Pass 0 to disable a fade-out effect that may be "
      "configured on the description.n"
      "Be aware that if a fade-out effect is used, the source will not immediately transtion to stopped state but "
      "will rather remain in playing state until the fade-out time has expired." )
{
   object->stop( fadeOutTime );
}

Again, the new macro allows us to get much more detailed with the documentation.

(continued in next post about addField)

#356 Member variables: addField(...)
Not a lot has changed here. When you are declaring variables for datablocks or objects, the same practice applies. We've just added a little more room for Doxygen tagging and using true types:

In C++:

addField( "coneOutsideVolume",   TypeF32,    Offset( mConeOutsideVolume, SFXDescription ),
	         "Determines the volume scale factor applied the a source's base volume level outside of the outer cone.n"
	         "In the outer cone, starting from outside the inner cone, the scale factor smoothly interpolates from 1.0 (within the inner cone) "
	         "to this value.  At the moment, the allowed range is 0.0 (silence) to 1.0 (no attenuation) as amplification is only supported on "
	         "XAudio2 but not on the other devices.nn"
	         "Only for 3D sound.n"
	         "@ref SFXSource_cones" );

The big change here is that you need to use a TYPEID<> macro for objects outside of your standard S32, F32, etc:

addField("emitter",           TYPEID< ParticleEmitterData >(),   Offset(emitterList,        SplashData), NUM_EMITTERS, "List of particle emitters to create at the point of this Splash effect.n");

While on the subject of variables, what about those pesky global variables defined in C++ but contain no real comments or docs? Well, we are covering those (slowly but surely) as well. The addVariable macro also takes advantage of our auto-Doxy system:

Con::addVariable( "SFX::numSources", TypeS32, &mStatNumSources, NULL,
	      "Number of SFXSources that are currently instantiated.\n"
	      "@ingroup SFX" );

So when we run the doc generator, the global variables defined in C++ are also placed in groups according to their module, such as SFX.

#357 Mich, you and your team are doing well with the docs. So far it's progressing well.

I am curious though if there are simple "move object" functions in TS though.

edit: to add: I have looked over a bit of the documentation, and it looks good. It's at least enough to start learning by knowing the functions. (and that's a good thing) Thanks a lot, and looking forward to more. ;)

#358 Callback functions, previously Con::executef(...)

Ohhh, the bane of documenting TorqueScript. Creating a callback for class objects, such as onAdd(...), onDamage(...), etc, was basically a single C++ call:

Con::executef( mDataBlock, "onLeaveLiquid", scriptThis(), mLiquidType.c_str() );

It works. It's easy. It's damn near impossible to auto-document this. Previously, it was a matter of just doing a Ctrl+F in the source for "Con::executef" and hand writing the docs that cover this. Doable, as it has been done before. Hooking up callbacks to the auto-documentation process had a fantastic side-benefit. In addition to documenting these chunks of code, we are implementing true callback systems. It's a three step process.

1. Declare a callback:

DECLARE_CALLBACK( void, applyDamage, ( const Point3F& hitPosition, const Point3F& hitNormal, SceneObject* hitObject ));

2. Implement the callback:

IMPLEMENT_CALLBACK( Lightning, applyDamage, void, ( const Point3F& hitPosition, const Point3F& hitNormal, SceneObject* hitObject ), 
												  ( hitPosition, hitNormal, hitObject ),
   "Informs an object that it was hit by lightning and needs to take damage.n"
   "@param Lightning object.n"
   "@param Object that was hit.n"
   "@param Position the lightning bolt hit in the world.n"
   "@param Range that the lightning bolt struck.n"
   "@see Lightning, LightningDatan"
);

Look at all that nice documentation. Lovely.

3. Finally, the actual function:

void Lightning::applyDamage( const Point3F& hitPosition,
                             const Point3F& hitNormal,
                             SceneObject*   hitObject)
{
   applyDamage_callback( hitPosition, hitNormal, hitObject );
}

Cleanly separated, define as you will, no hidden injected Con::executef(...). My hats off to Rene and Tom for this. I love it, not just for the sake of documentation, but for the new flexibility in expanding the engine.

#359 What about the often forgotten enumeration types? Surely people have come across a few and not known what's really going on. Well, we have that covered now as well. This is similar to the callback setup in that it's a three part process.

1. The enumerator definition:

/// Rolloff curve used for distance volume attenuation of 3D sounds.
enum SFXDistanceModel
{
   SFXDistanceModelLinear,             ///< Volume decreases linearly from min to max where it reaches zero.
   SFXDistanceModelLogarithmic,        ///< Volume halves every min distance steps starting from min distance; attenuation stops at max distance.
};

2. Declare the public type bits:

DefineEnumType( SFXDistanceModel );

3. Implement and document:

ImplementEnumType( SFXDistanceModel,
   "Type of volume distance attenuation curve.\n"
   "The distance model determines the falloff curve applied to the volume of 3D sounds over distance.\n\n"
   "@ref SFXSource_volume\n\n"
   "@ingroup SFX" )
   { SFXDistanceModelLinear, "Linear",
      "Volume attenuates linearly from the references distance onwards to max distance where it reaches zero." },
   { SFXDistanceModelLogarithmic, "Logarithmic", 
      "Volume attenuates logarithmically starting from the reference distance and halving every reference distance step from there on. "
      "Attenuation stops at max distance but volume won't reach zero." },
EndImplementEnumType;

Again, wonderful. From a documentation perspective, this is great for a writer. See the code, document the code.

#360 And finally, for the 300 lb gorilla in the room: Object Documentation. Ok, you know how to call a function because you read the syntax guidelines. You know what the function does, because you looked it up in the reference. You know what variables the function takes, again because of the reference material

But what about the actual object? What good is calling a function on an object you don't know about. In comes my favorite new macro: ConsoleDocClass. Previously...well, there was just raw Doxygen chunks (which didn't hook up to the script API) or just hand writing everything. Both are doable, but with this great new automated system...why not give object documentation its own shining spot.

ConsoleDocClass(...)
From here on, if a programmer creates a an object that is exposed to TorqueScript he better damn well at least write a ConsoleDocClass for it. Much pain will be brought down otherwise.

On that note, how does it work? =). Using standard Doxygen tags and this macro, you can document the object in the same source file it is written:

ConsoleDocClass( SFXAmbience,
	   "@brief A datablock that describes an ambient sound space.\n\n"
	   
	   "Each ambience datablocks captures the properties of a unique ambient sound space.  A sound space is comprised of:\n"
	   
	   "- an ambient audio track that is played when the listener is inside the space,\n"
	   "- a reverb environment that is active inside the space, and\n"
	   "- a number of SFXStates that are activated when entering the space and deactivated when exiting it.\n"
	   "\n"
	   
	   "Each of these properties is optional.\n\n"
	   
	   "An important characteristic of ambient audio spaces is that their unique nature is not determined by their location "
	   "in space but rather by their SFXAmbience datablock.  This means that the same SFXAmbience datablock assigned to "
	   "multiple locations in a level represents the same unique audio space to the sound system.\n\n"
	   
	   "This is an important distinction for the ambient sound mixer which will activate a given ambient audio space only "
	   "once at any one time regardless of how many intersecting audio spaces with the same SFXAmbience datablock assigned "
	   "the listener may currently be in.\n\n"
	   
	   "Each SFXAmbience instance will automatically add itself to the global SFXAmbienceSet.\n\n"
	
	   "At the moment, transitions between reverb environments are not blended and different reverb environments from multiple "
	   "active SFXAmbiences will not be blended together.  This will be added in a future version.\n\n"
	   
	   "@tsexample\n"
	   "singleton SFXAmbience( Underwater )\n"
	   "{\n"
	   "   environment = AudioEnvUnderwater;\n"
	   "   soundTrack = ScubaSoundList;\n"
	   "   states[ 0 ] = AudioLocationUnderwater;\n"
	   "};\n"
	   "@endtsexample\n"
	      
	   "@see SFXEnvironment\n"
	   "@see SFXTrack\n"
	   "@see SFXState\n"
	   "@see LevelInfo::soundAmbience\n"
	   "@see Zone::soundAmbience\n"
	   "@ref Datablock_Networking\n"
	   "@ingroup SFX\n"
	   "@ingroup Datablocks\n"
	);

This will automatically be placed in the SFX and Datablocks modules of the TS Reference Manual chm. You have links to other classes, a description, and even a basic example. Honestly, if I'm going to be documenting a class, this is the way I want to do it. In fact, this whole system is how I like it...for three reasons:

1. It's not just the people reading the script manual who are getting the docs here. A C++ programmer is going to see this in the source code while they are browsing. Documentation in two spots? Win!

2. It's automated. I run a single batch file to dump out the CHM for all the stuff I've been talking in about in the past few posts.

3. Accountability. It's a lot easier for me to ask the programmers to write docs if it's in their territory. They are already in the code and they know about the code they are writing. This is a far more attractive proposition to a programmer than hand writing the documentation in Word, Html, or even a Wiki.

Anyone ever said "Show me how to make a programmer write docs and I'll _____". Well, point them to this thread next time. Because it's happened. That's one of the best thing to come out of this drive for new docs. While the Torque engine teams are slammed with bug fixing, polishing, and features, everyone has shown a genuine concern and desire for better documentation. Everyone has chipped in, and they all deserve a huge "Kudos" for the effort.

This project is 10 years in the making. People in this thread are very frustrated and have lost some faith. I see that, and I kind of understand it. It's true you have heard more or less the same phrase over the years. 10 years and classes like Lightning and Debris have not been fully documented? Yeah, big WTF.

Does it suck that all that has built up and fallen in the current team's lap, with us taking the heat? Yeah, it does. Well, here's the thing. Rather than me saying one more time: "It's ok, we are working. Believe us"...I pushed the new docs online, uploaded the latest CHM, and spent the past hour writing these several chained posts to try and gain a sliver of faith back from you all by showing the actual progress.

You wanted our ears, so you posted. You wanted our words, so we responded. You wanted honesty, and we gave it. You wanted proof? Well, here it is.

Why this huge crazy string of posts? I'm going heads down for the next two weeks. I will not be in the forums or IRC. It's the big crunch...the final push. There is quite a bit left to do to finish off the Script API docs, and I have to block everything else out to get it done.

So, do not think I've blocked this thread out because I don't like you all or don't want to hear the feedback. Done with words, time for the docs.