Headless Client - server stress testing

#1 It's the filesystem crapping up with several executables trying to re-generate the procedural shaders. If you have all your shaders generated, add this to your main.cs:

$ShaderGen::GenNewShaders = false;

It'll disable shader generation and will use whatever is already in the shaders/procedural folder.

As for not opening windows, that'd be a bit hard. You'd need to prevent the canvas from being created, but I'm not sure if a client can work without a canvas, since some of the level loading logic depends on GUIs (which isn't the case for dedicated servers).

#2 Thanks Manoel,

That solved the issue. Still, with the rather large level we had I was only able to get 84 clients running at once before my 6 GB, Windows 7 (64-bit) machine got starved for resources and was unable to launch any more. Just running those 84 processes really brought the OS to its knees. ;) Then again I'm only running an 2.8GHz Intel Core Duo - I'm trying to convince my boss that I need an 8-core i7 system with 16 GB or RAM... yeah, that's the ticket! :D

= Ed =

#3 Yea... get an i7 ASAP. Its a wonderful feeling to see all 8 cores pegged for a minute or two while compiling Torque. :)

Quote:It's the filesystem crapping up with several executables trying to re-generate the procedural shaders.

I have a bug report in the system for me to revive the binary shader cache so that new shaders are not always rebuild and recompiled.

That said... it shouldn't be making materials in the first place in a real headless render. Even with the client running on the NULL device i don't think it just skips over rendering all together. So it goes thru the motions of rendering only to get to a empty drawPrim() function. I bet we could better optimize that case by either not creating the canvas or at least skipping over the top level render call.

Sounds like a fun little project. :)

#4 Because of my custom renderDelegate, and the fact that I don't know how to tell (on the C++ side) if I'm running headless or not, I had to solve some of my own issues with a conditional compile to create a special, headless version. Is there some way that I can easily tell if my cliient is running headless?

= Ed =

#5 The easiest way to tell is to run a debug build and set a breakpoint in your renderDelegate. I would think that ideally none of the rendering side should be called to get optimal performance in a headless client case.

If it hits i would go up the callstack and comment out the top level rendering and see if your headless client seems to still run correctly.

If you get any fixes to make headless clients work well let me know and i'll merge them into the core.

#6 At present we are setting a variable in our script that we can test on the C++ side, so that seems to work for us.

As for fixes for headless clients to work, I have 3 at the present time. These are all fixes to version 1.0.1 so your code base will probably differ.

1) in core/scripts/client/lighting.cs, in the initLightingSystems() function, I enclosed the calls to set up a Light Manager only if not running headless. Here is the new code snippet:

if ( !$headless )       // don't try to set up a light manager when headless
   {  
      // Try the perfered one first.
      %success = setLightManager( $pref::lightManager );
      if ( !%success )
      {
         // The perfered one fell thru... so go thru the default
         // light managers until we find one that works.
         %lmCount = getFieldCount( $pref::defaultLightManagers );
         for ( %i = 0; %i < %lmCount; %i++ )
         {         
            %lmName = getField( $pref::defaultLightManagers, %i );
            %success = setLightManager( %lmName );
            if ( %success )
               break;
         }
      }
      
      // Did we completely fail to initialize a light manager?   
      if ( !%success )
      {
         // If we completely failed to initialize a light 
         // manager then the 3d scene cannot be rendered.
         quitWithErrorMessage( "Failed to set a light manager!" );
      }
   }

2) in Engine/source/T3D/levelinfo.cpp I made a minor change to LevelInfo::-updateSceneGraph(). Towards the end of that function, there is a call to get the LightManager pointer if this is a client object. Immediately after this call is an attempt to use this LightManager. I put in a safety check to validate that the pointer is not NULL. This is a safety check that should be there in any event. Here is the updated code snippet:

#ifndef TORQUE_DEDICATED
   if(isClientObject())
   {
      LightManager* manager = gClientSceneGraph->getLightManager();
	  if ( manager )
		_onLMActivate(manager->getId(), true);
   }
#endif

3) Finally is the fix mentioned my Manoel in this thread. I added his command in the section of game/scripts/main.cs that handles the -headless command line input switch. Snippet follows from function parseArgs() in that file:

case "-headless":
            enableWinConsole(true);
            $headless = true;     
            $ShaderGen::GenNewShaders = false;

I hope this helps.

= Ed =
[code]