Gathering a Depth Map?

#1 @Matt

Here is an overview of a solution which Pat clued me into:

1. Create a new RenderElemMgr for rendering the zpass data and have it process after RIT_Sky and before RIT_Object.
2. Setup a special hlsl pixel and vertex shader which purely writes the world space z distance from the camera to the render target.
3. In RenderInstManager::addInst() redirect appropriate RenderInsts to your new zpass manager.
4. In your new zpass manager setup a render to texture target to gather your z data.
5. When your new zpass manager render() call is made push your render target then render all the instances.
6. Extend CustomMaterial to attach the new zdepth texture when it sees a texture called $depth (just like it does for $fog and $cubemap.

There are too many details and i don't have time to dive into here, but this is the jist of it.

Anyone in the community looking into this and willing to contribute code?

#2 Here is an example of a trick done with a z depth map.


In the pixel shader i used the distance between the scene depth and the water surface to decide how clear to make the water. This gives me nice transparency in the shallows and along the edges of dynamic objects moving thru the water.

As a side effect it also hides all z-fighting you typically see on shore lines.

#3 Fantastic overview (and pretty picture!)

One additional note: Make sure, in the vertex shader, you pass both Z and W to the pixel shader, and divide z by w in the pixel shader, not the vertex shader, otherwise you will get incorrect results because interpolation is working against you, instead of for you.

#4 @Pat - Actually i found that doing that for my case was bad.

I tried to work with the 0 to 1 depth generated by doing z/w. That made the depth comparison for water inaccurate as i would lose precision the farther away the pixel was in the scene. This is because the z buffer value isn't linear.

I instead store just the z distance of the pixel from the camera near plane. I figure that whatever i ever needed to do in z/w space i can also do with more precision with simple z.

#5 Awesome. That's a nice step-by-step instruction list. When GFX->pushActiveRenderSurfaces(); is called, that doesn't clear the render surface, does it? Theoretically I could push and pop it each time a new object is rendered without clearing the existing depth data, correct? Also, I've seen several HLSL depth implementations. Is there any reason why z is divided by w?

#6 Ok, steps 1-3 are really easy to add. However, I'm a bit confused on how my Zpass and Z render function should look like. So far this is what my code looks like. Am I doing anything illogical or am I on the right track? Also, could you elaborate more on the Custom Material idea? I'm not really interested in depths for each individual object, but more for gathering it for the whole scene for such techniques as Depth of Field, Motion Blur, and Screen-Space Ambient Occlusion.

The code is based on the renderGlowMgr.

//-----------------------------------------------------------------------------
// Torque Game Engine Advanced
// Copyright (C) GarageGames.com, Inc.
//-----------------------------------------------------------------------------
#include "renderDepthMgr.h"
#include "materials/sceneData.h"
#include "sceneGraph/sceneGraph.h"
#include "materials/matInstance.h"
#include "../../example/shaders/shdrConsts.h"


//**************************************************************************
// RenderDepthMgr
//**************************************************************************

//-----------------------------------------------------------------------------
// render
//-----------------------------------------------------------------------------
void RenderDepthMgr::render()
{
   if(!mFirstRun)
   {
   mDepthShader = dynamic_cast<ShaderData*>(Sim::findObject( "DepthShader" ) );
   mDepthShader->initShader();
   GFXVideoMode mode = GFX->getVideoMode();
   mDepth.set( mode.resolution.x, mode.resolution.y, GFXFormatR8G8B8A8, &GFXDefaultRenderTargetProfile, 1 );
   mFirstRun = 1;
   }

   RectI vp = GFX->getViewport();

   GFX->pushActiveRenderSurfaces();
   GFX->setActiveRenderSurface( mDepth );
   GFX->clear( GFXClearTarget, ColorI(0,0,0,0), 1.0f, 0 );

   GFX->pushWorldMatrix();

   // set render states
   GFX->setCullMode( GFXCullCCW );
   GFX->setZWriteEnable( false );

   // init loop data
   GFXVertexBuffer * lastVB = NULL;
   GFXPrimitiveBuffer * lastPB = NULL;
   SceneGraphData sgData;
   U32 binSize = mElementList.size();

   for( U32 j=0; j<binSize; )
   {
      RenderInst *ri = mElementList[j].inst;

      U32 matListEnd = j;

         U32 a;
         for( a=j; a<binSize; a++ )
         {
            RenderInst *passRI = mElementList[a].inst;

            // fill in shader constants that change per draw
            //-----------------------------------------------
            GFX->setVertexShaderConstF( 0, (float*)passRI->worldXform, 4 );

            // set object transform
            MatrixF objTrans = *passRI->objXform;
            objTrans.transpose();
            GFX->setVertexShaderConstF( VC_OBJ_TRANS, (float*)&objTrans, 4 );
            objTrans.transpose();
            objTrans.inverse();
			
			MatrixF proj = GFX->getProjectionMatrix();
			MatrixF world = GFX->getWorldMatrix();
			proj.mul(world);
			proj.transpose();
			GFX->setVertexShaderConstF( VC_WORLD_PROJ, (float*)&proj, 4 );
            // fill in eye data
            Point3F eyePos = gRenderInstManager.getCamPos();
            objTrans.mulP( eyePos );
            GFX->setVertexShaderConstF( VC_EYE_POS, (float*)&eyePos, 1 );
			mDepthShader->shader->process();
            // set buffers if changed
			   if(passRI->primBuff)
			   {
               if( lastVB != passRI->vertBuff->getPointer() )
               {
                  GFX->setVertexBuffer( *passRI->vertBuff );
                  lastVB = passRI->vertBuff->getPointer();
               }
               if( lastPB != passRI->primBuff->getPointer() )
               {
                  GFX->setPrimitiveBuffer( *passRI->primBuff );
                  lastPB = passRI->primBuff->getPointer();
               }

               // draw it
               GFX->drawPrimitive( passRI->primBuffIndex );
			   }
			   GFX->disableShaders();
         }
         matListEnd = a;

      // force increment if none happened, otherwise go to end of batch
      j = ( j == matListEnd ) ? j+1 : matListEnd;

   }

   // restore render states, copy to screen
   GFX->setZWriteEnable( true );
   GFX->popActiveRenderSurfaces();
   //glowBuffer->copyToScreen( vp );

   GFX->popWorldMatrix();
}

#7 That looks about right.

One thing that you might want to do is clear the z and stencil on your clear as well and fill it with white (which will mean its filled with the farthest value and not 0 the nearest).

My zdepth vert shader does this...

// Some shaders hide data in W... in our
   // case we just need a pure transform.
   float4 inPos = IN.pos;
   inPos.w = 1.0;
 
   // Normal transform stuff.
   OUT.hpos = mul( modelViewProj, inPos );
 
   // Pass the transformed position to the pixel shader
   // in a texcoord register... it doesn't like it otherwise.
   OUT.pos = OUT.hpos;

My pixel shader is simple too...

// Clip any negative z pixels behind the camera.
   clip( IN.pos.z );
 
   // We just want the world space z distance from 
   // the camera near plane.... z/w is non-linear and
   // sucks as far as i can tell.
   float OUT;
   OUT = IN.pos.z;
 
   // Our render target is a single channel float format, 
   // so we just need to return a single value.
   return OUT;

... and that's it.

#8 You DO want to render to the Zbuffer, so turn on ZWrites.

This is actually a useful technique for more than just getting depth information. The way I have the render-manager set up, it will either output to a depth target, or disable color writes, and just populate the depth buffer. This will ensure that you have 0-overdraw on your opaque objects. This is very useful for fillrate limited games which have expensive shaders, and overdraw issues.

For example: In Torque the Mesh and Interior objects have different render managers. These objects are sorted front-to-back within their respective render managers, but you can still have a case where a large mesh object occludes an interior, causing a bunch of overdraw, or vice-versa.

The basic idea of eliminating overdraw via a z-fill pass is this:
-Collect, all opaque objects in the scene
-Disable color writes so that the color buffer is not modified, turn on z-writes
-Render all opaque objects as inexpensively as possible (shaders that Tom posted, for example, are cheap)
-Re-enable color writes, DISABLE z-writes (they have already written Z), make sure z-test is enabled
-Render all opaque objects as normal. There will be 0 overdraw on them.

This render manager was written for Legions which has large interiors, large static TS meshes, and some very expensive shaders. Eliminating overdraw was the primary concern, collecting depth information was the secondary objective. Previously I implemented depth maps via MRTs by hacking stuff into ShaderGen. This was not the way to go. MRTs are a pain in the ass. They shouldn't be, but they are.

The maddening thing, in all of this, is that the card HAS the depth information, and we just can't sample it in DX9.

#9 So is a depth-shader even needed for this? I'm still having a bit of trouble getting this working, but part of the problem is simply my lack of understanding with the GFX layer. You shouldn't even need a shader if you're just saving what the Z-buffer writes, correct?

#10 Matt, that's only partially true. It depends on what you want to do with the information. If you just want to use it to eliminate overdraw, than you don't need a shader or a render target. (You don't want a render target actually) If you want to sample from the depth information than you do need a shader and a render target.

Also, if you want to sample from the depth AND you want to eliminate overdraw, bind the back-buffers' z-buffer to the z-buffer for the draw, and then the sampleable depth buffer to Color0. Then the shader outputs depth to color0, and the z-buffer still gets populated.

#11 Hey guys,

I hate to dredge up old posts, but I have a burning question on this topic. Is it possible to make this work for terrain? It seems RenderInstManager doesn't handle terrain at all, so the above method doesn't produce depth data for it? (I'm using TGEA 1.7.0)

Thanks,
Lorne

#12 Yikes..kind of a mess of code from all those months back. It is definitely possible to gather depth information for the terrain. All that is needed is another render pass of the terrain using a depth shader and linking it to the object depth render target and depth texture.

#13 Like Matt said, you can get the depth info. You need to add support for RIT_Objects to your depth manager. Then iterate through those render instances seperate from the RIT_Mesh / RIT_Interior instances. Then you need to hack up the terrain to render using your shader.

For the research codebase GG studios is using, I changed the RIT_Object callback to accept an override material which would be used in place of the material that the object normally renders with. That way you could pass a depth material into the terrain code and have it use that. It was a bit of work to get the terrain code to render with materials, so it's probably quickest to hack it with some statics. ;)

#14 My old method was a bit unoptimized, but basically I copied the for loop used to render all the mDiffusePasses in the chunkBatcher and setup my render targets before and after the for loop, while having it execute the depth shader (tied in with a precomputed normal map for deferred rendering, seeing as Legacy and Atlas normals seem to be "damaged").

#15 Thanks for the help guys. I'm starting to get somewhere.... just one more question ;)

Any idea why Anti-Aliasing messes up my depth-gathering pass? I've read that AA is incompatible with multiple render targets, but AFAIK I'm not using that. Visually, the effect is as if GFX->setZEnable(false) has been called, so the first geometry to be rendered is what's displayed - i think.

I'm setting up my render like so:

GFXTextureTargetRef depthTextureTargetRef = GFX->allocRenderToTextureTarget();
GFXTexHandle depthTexHandle = GFXTexHandle(resolution.x, resolution.y, GFXFormatR8G8B8A8, &GFXDefaultRenderTargetProfile, 1 );
mDepthTextureTargetRef->attachTexture(GFXTextureTarget::Color0, depthTexHandle );
mDepthTextureTargetRef->attachTexture(GFXTextureTarget::DepthStencil, GFXTextureTarget::sDefaultDepthStencil );
GFX->setActiveRenderTarget( depthTextureTargetRef );
GFX->clear( GFXClearTarget, ColorF(1, 1, 1, 1), 1.0f, 0 );

#16 I haven't experimented much with anti-aliasing, but I believe by attaching another texture, mDepthTextureTargetRef->attachTexture(GFXTextureTarget::DepthStencil, GFXTextureTarget::sDefaultDepthStencil ); for example, you are doing a multiple render target scenario, where it needs to output color to TWO renderable textures.

#17 Change this:

GFXTexHandle depthTexHandle = GFXTexHandle(resolution.x, resolution.y, GFXFormatR8G8B8A8, &GFXDefaultRenderTargetProfile, 1 );

to this:

GFXTexHandle depthTexHandle = GFXTexHandle(resolution.x, resolution.y, GFXFormatR8G8B8A8, &GFXDefaultRenderTargetZBufferProfile, 1 );

When using the GFXTextureTarget::sDefaultDepthStencil as the zbuffer target, you need to make sure your color target is created with compatible parameters. The GFXDefaultRenderTargetZBufferProfile forces the render target to use the same creation parameters as the backbuffer. When everything matches up, the zbuffer will work for you. This was the issue with the glow buffer in older releases of TGEA.

#18 Ah yes, I tried that one... All I get is garbage (random textures from memory) on my screen when I display the texture. It doesn't make much sense to me as I'm calling GFX->clear (and then theoretically rendering a bunch of stuff to it).

#19 Well the other issue is that it's performing the AA on the depth data, because it's a color buffer. You won't be able to use a R32F texture with MSAA on all hardware, so R8G8B8A8 is still your best bet, but the data is going to get pretty well hosed by the AA.

This is one of the things that DX10 is good for. It lets you control MSAA much better.

#20 Awesome! Pat, you just solved my problem... months ago in another thread...

"You don't want to call 'clearAttachments()' before you pop the active render target."

...And guess what I was doing! Thanks.