Prev/Next Blog
by date

Detail maps - fixed resource

Name:	Apparatus
Date Posted:	Jun 22, 2008
Rating:	4.4 out of 5
Public:	YES
Comments:	YES
RSS Feed:	or Subscribe with .
Profile Page:	View profile page for Apparatus

Blog post

Ok guys, looks like we fixed the detail map issue in TGE, as described here and here. Now, can you please post a list with the code changes so that everyone can take advantage of it?

Thanks!

Recent Blog Posts

List:	11/04/08 - Kork 2.0, halted work 09/24/08 - Constructor Pipeline (I) 09/23/08 - Interior Guide Pack Free Download 09/23/08 - Kaboom! Pack for TGEA 09/15/08 - About Constructor 09/11/08 - Interior Guide Pack Released 09/10/08 - Associate Rant 09/08/08 - Interior Guide Pack update

Submit your own resources!

Joseph Greenawalt (Jun 22, 2008 at 23:31 GMT)

Here's the potential fix that I came up with:

Change the block of code beginning at line 1559 of tsshapinstance.cc from:

   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
      mCurrentIntraDetailLevel = 0.0f;
      return mCurrentDetailLevel;
   }

to:

   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
      mCurrentIntraDetailLevel = 1.0f - errorTOL;
      mCurrentIntraDetailLevel = mCurrentIntraDetailLevel>1.0f ? 1.0f : (mCurrentIntraDetailLevel<0.0f ? 0.0f : mCurrentIntraDetailLevel);
      return mCurrentDetailLevel;
   }

And here's Morrock fix:

Quote:
I added the following code to the S32 TSShapeInstance::selectCurrentDetailEx(F32 errorTOL) function, right at the beginning after the description comment.

if (mShape->mSmallestVisibleDL == 0)
   {
	   // get here if we are drawing at DL==0
	   mCurrentDetailLevel = 0;
	   mCurrentIntraDetailLevel = 1.0f;
}

The main difference seems to be that Morrock's fix does not fade out the detail maps as distance from the object decreases. Basically I was trying to make my fix work as closely as possible to the other detail selection method that I mentioned in the thread (the one that's working already), including the fading of detail maps with distance. Not fading detail maps does prevent the "popping" issue that I mentioned. I'm not really sure that the fading is doing too much visually, so it might actually be better to just set mCurrentIntraDetailLevel to 1.0 for the last DL regardless of the distance from the object as Morrock suggests. I haven't tried Morrock's fix here yet, but it looks to me like the mCurrentIntraDetailLevel value set there is going to be overwritten by the code below? Is there supposed to be a return statement in that "if" block?
Edited on Jun 22, 2008 23:33 GMT

Joseph Greenawalt (Jun 22, 2008 at 23:44 GMT)

Here's a hybrid of the two. I'm not sure how the code Morrock posted is working since the very next "if" block immediately overwrites the values that he's setting at the beginning of the function. I think this is probably closer to what was intended?

Change the block of code beginning at line 1559 of tsshapinstance.cc from:

   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
      mCurrentIntraDetailLevel = 0.0f;
      return mCurrentDetailLevel;
   }

to:

   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
      mCurrentIntraDetailLevel = 1.0f;
      return mCurrentDetailLevel;
   }

Edited on Jun 22, 2008 23:45 GMT

Apparatus (Jun 22, 2008 at 23:49 GMT) Resource Rating: 5

Now this is GG community at its best; less than 12 hours ago posted, now fixed; you can test this on your own or you can wait my next pack for a full blown preview of this resource in action :)

Thanks everyone!

Morrock (Jun 22, 2008 at 23:53 GMT)

Whoops, I swear I copied that return line. I actually had to recopy it 3 times because I kept leaving out the function name or the end bracket... I was in a rush to get out the door. My actual code was as follows. It only took effect if the smallest DL we are working with == 0, so it is a check at the beginning of code to try and establish that block that can be found (but never accessed) at the end of the block. Here is my code with a few lines around it

// deal with degenerate case first...
   // if smallest detail corresponds to less than half tolerable error, then don't even draw
	if (mShape->mSmallestVisibleDL == 0)         //added
   {                                                   //added
	   // get here if we are drawing at DL==0   //added
	   mCurrentDetailLevel = 0;                 //added
	   mCurrentIntraDetailLevel = 1.0f;        //added
	   return mCurrentDetailLevel;             //added
   }                                               //added
	F32 prevErr;
   if (mShape->mSmallestVisibleDL<0)
      prevErr=0.0f;
   else
<keeps going...>

It only works if there is only one detail level (DL == 0) so I don't think it would be possible to fade out the detail level anyways, would it...?
Edited on Jun 23, 2008 01:13 GMT

Joseph Greenawalt (Jun 22, 2008 at 23:58 GMT)

It's not possible to fade out the last detail level (guess it is, but it would require other code changes and wouldn't be desirable, I think), but the detail maps themselves can and do fade out for the last detail level as the distance increases when using the other detail level selection method that I mentioned here: www.garagegames.com/mg/forums/result.thread.php?qt=76451

Like I said though, I think your fix may be better since it does prevent the "popping" effect and you really can't see the effects of the detail map fade-out anyway unless you look very very closely :-)

Morrock (Jun 23, 2008 at 00:04 GMT)

Ah, I think I get what you're saying now. So even if there is only one detail level to use (in the case of my code, 0) the detail map can still fade even if the shape level of detail doesn't? Well then, I think I'd like to find a fix to that if I can. I also cleared up a misconception in the other thread about "no noticable difference", I was actually talking about combining yours and my resource.

And ooh, bridge pack, thank you! I would love to get my hands on that mountain/environment pack now when it is done, it looks great (whatever the official name is, the thing with the great looking waterfalls and rivers!)
Edited on Jun 23, 2008 00:04 GMT

Joseph Greenawalt (Jun 23, 2008 at 00:20 GMT)

Thanks Apparatus, I really appreciate the models :-) Your artwork puts my puny attempts at "art" to shame (bigtime) :-) I've also learned quite a bit that should be helpful to me in my Blender exporter work. I didn't know that the blender-exported models were using a different detail level selection method than 3dsmax-exported models until now. I've also got a good start now at getting tiled/scaled detail maps working properly with the blender exporter.

@Morrock:
Yeah, no problem on the misunderstanding. I was just worried that I had screwed up and missed something :-)

As far as the detail map fading goes, I'll probably be looking for a fix for the popping/disappearing lightmap problem soon since it affects Blender-exported models as well. As a first guess, I'd say it might have something to do with the material setup code running out of texture units or something getting swapped that shouldn't be.

Morrock (Jun 23, 2008 at 00:32 GMT)

No... I don't think that's it. I don't know if I'm getting this right, but do IntraDetailLevel's work that 1.0f = full render, 0.0f = no render, and in between is fading? If so, wouldn't we need some kind of dynamic ratio between player distance and object LOD?
In your code:

mCurrentIntraDetailLevel = mCurrentIntraDetailLevel>1.0f ? 1.0f : (mCurrentIntraDetailLevel<0.0f ? 0.0f : mCurrentIntraDetailLevel);

Doesn't this either check if detail level is greater than 1, if so set it to 1. Else, if detail level is less than 0, set it to 0, other wise, just keep it as mCurrentIntraDetailLevel - errorTol; ? Unless errorTol changes based on the players distance away from the object, I don't see how well this could work. I see what you're going for, and I also tried to replicate the selectCurrentDetail(F32 size) function, but since I had no idea what F32 size could equal, I couldn't do it. At least you got it going forward. :-)

I tried this code, and it actually seems to work. I really don't even know how, I was just playing with it. Hehe.

if (mShape->mSmallestVisibleDL == 0)
   {
	   // get here if we are drawing at DL==0
	   mCurrentDetailLevel = 0;
	   Box3F boundBox = mShape->bounds;
	   F32 estimSize = boundBox.max.len();
	   mCurrentIntraDetailLevel = mShape->mSmallestVisibleSize / estimSize;
	   return mCurrentDetailLevel;
   }

Am I just seeing things that I want the detail map to fade out, or is it actually doing it? Hard to tell at the low-res I've got. Oh, and you need #include "math/mBox.h" at the beginning of the file.

Oh, and Apparatus, I just finished downloading the bridges. They look great, man! Thanks!
Edited on Jun 23, 2008 01:16 GMT

Joseph Greenawalt (Jun 23, 2008 at 00:47 GMT)

Quote:
Doesn't this either check if detail level is greater than 1, if so set it to 1. Else, if detail level is less than 0, set it to 0, other wise, just keep it as mCurrentIntraDetailLevel - errorTol; ? Unless errorTol changes based on the players distance away from the object, I don't see how well this could work.

errorTOL does change with the camera/viewpoint distance from the object. Start a new mission with only one dts file and try setting a breakpoint there while a debug instance is running. Check the value, remove the breakpoint and continue. Move the camera closer, set breakpoint, check the value, rinse repeat :-)

Quote:
I see what you're going for, and I also tried to replicate the selectCurrentDetail(F32 size) function, but since I had no idea what F32 size could equal, I couldn't do it. At least you got it going forward. :-)

Yeah, it's not very scientific, but I did an "eyeball" test using breakpoints and a blender-exported model (which uses the other detail selection method) and it's pretty close, visually at least :-)

Quote:
Am I just seeing things that I want the detail map to fade out, or is it actually doing it?

It is fading out, at least it is for me. It's easier to see with a high-contrast texture such as a checkerboard pattern or the "smiley" cube that I posted in Apparatus's other plan (you can dl it if you want and take a look). Set the camera speed to "very slow" and slowly back away and you should see it. Mipmapping does make it a bit hard to see, but it really is fading out. Whether it needs to fade out or not is an open question though :-)

Morrock (Jun 23, 2008 at 01:02 GMT)

So it really is fading out? Awesome. I didn't believe it was (it never works that easily!) You are talking about the code I just post right? :-) Well, I started writing another function anyways, this one may be better or it might not be, but here is what I have so far, it can see this one fading (I had to go fullscreen and crank the resolution, despite the lag) though it doesn't seem fully detailed when I'm nearly pressed against the surface.

   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
	  F32 currSize = mShape->details[mCurrentDetailLevel].size;
	  F32 nextSize = mCurrentDetailLevel==0 ? 2.0f * currSize : mShape->details[mCurrentDetailLevel - 1].size;
	  mCurrentIntraDetailLevel = nextSize-currSize > 0.1f ? (currSize - errorTOL) : 1.0f;
	  mCurrentIntraDetailLevel = mCurrentIntraDetailLevel>1.0f ? 1.0f : (mCurrentIntraDetailLevel<0.0f ? 0.0f : mCurrentIntraDetailLevel);
      return mCurrentDetailLevel;
   }

It's much more similar to the other functions and doesn't require the if (mShape->mSmallestDetailLevel ==0) check at the beginning. Then again, I guess I could just put that code I posted into the other if blocks as well...

And thanks, I'll download your material cube and try.

Edit: wow that cube is useful!
Edited on Jun 23, 2008 01:10 GMT

Joseph Greenawalt (Jun 23, 2008 at 01:13 GMT)

Quote:
So it really is fading out? Awesome. I didn't believe it was (it never works that easily!) You are talking about the code I just post right? :-)

No, I was speaking in general about the detail map fading (sorry, I misunderstood again :-).

Quote:
Well, I started writing another function anyways, this one may be better or it might not be, but here is what I have so far, it can see this one fading (I had to go fullscreen and crank the resolution, despite the lag) though it doesn't seem fully detailed when I'm nearly pressed against the surface.

The value of mCurrentIntraDetailLevel might not ever be reaching 1.0 (fully visible) or might be approaching it too slowly. I had that problem with a couple of the solutions that I tried.

I'll take a look tomorrow and see if I can provide confirmation that your new code is working. Right now I've got a couple of high-priority things that I need to get done before tomorrow morning, so I'll have to put off testing until that's finished :-(

/off to work for now

Apparatus (Jun 23, 2008 at 01:17 GMT) Resource Rating: 5

Ok, here's another model I want show off in order to stress the relevance of detail maps. Not only it adds some detail to the model but to me it looks like the lighting overall is improved;

without

and with detail map

judge for yourselves.

Morrock (Jun 23, 2008 at 01:17 GMT)

Ah well then. Thanks for the help tonight. I would just ignore that first solution, but the previous one based off the other functions seems like it's working. I'll do what I can to get it to reach 1.0;

@App, it does very well look like it improves the lighting quality. I also noticed this with the big rock. I'm not sure if it really does, but it certainly gives it the effect (which I suppose is what counts.) I think I just got the solution, I noticed in my earlier example I left out a division expression which should have been there. Currently testing! :D

news edit: Oh... so close. I nearly got it, but I somehow got it so the detail level increases as you get further away from the object, not closer :( Almost there!
Edited on Jun 23, 2008 02:44 GMT

Morrock (Jun 23, 2008 at 03:04 GMT)

Well, I got good news and bad news. Good news, I got code that works! The bad news, as soon as the mCurrentIntraDetailLevel value get's below 0.45 the detail map seems to disappear completely. I'm not sure if this is a problem within the engine itself, or something with the model. mCurrentIntraDetailLevel does however get progressively smaller as you get farther from the object. From about 0-10yards from the object it = 1.0 (perfect detail). After 10 yards, it takes about 35 yards to reach 0.45. And after you reach about 80 yards away, the detail = 0.0 (none)

I'm still not entirely sure how detail maps themselves work (though I do understand the fade progression now and sizes) so this could just be an exporter error, or maybe a missed setting. Or something with the engine. Either way I'm out for tonight so I'll continue tomorrow. But here is the code I used.

Starting from line 1554 "// if smallest detail corresponds to less than half tolerable error [...]" and down

if (mShape->mSmallestVisibleDL == 0)
   {
	   // get here if we are drawing at DL==0
	  mCurrentDetailLevel = 0;
	  F32 currSize = mShape->details[mCurrentDetailLevel].size;
	  F32 nextSize = 2.0f * currSize;
	  F32 size = currSize / errorTOL;
	  mCurrentIntraDetailLevel = (size - currSize) / (nextSize - currSize);
	  mCurrentIntraDetailLevel = mCurrentIntraDetailLevel>1.0f ? 1.0f : (mCurrentIntraDetailLevel<0.0f ? 0.0f : mCurrentIntraDetailLevel);
	  return mCurrentDetailLevel;
   }
	F32 prevErr;
   if (mShape->mSmallestVisibleDL<0)
      prevErr=0.0f;
   else
      prevErr = 10.0f * mShape->details[mShape->mSmallestVisibleDL].averageError * 20.0f;
   if (mShape->mSmallestVisibleDL<0 || prevErr<errorTOL)
   {
      // draw last detail
      mCurrentDetailLevel=mShape->mSmallestVisibleDL;
	  F32 currSize = mShape->details[mCurrentDetailLevel].size;
	  F32 nextSize = mCurrentDetailLevel==0 ? 2.0f * currSize : mShape->details[mCurrentDetailLevel - 1].size;
	  F32 size = currSize / errorTOL;
	  mCurrentIntraDetailLevel = nextSize-currSize > 0.1f ? (size-currSize) / (nextSize-currSize) : 1.0f;
	  mCurrentIntraDetailLevel = mCurrentIntraDetailLevel>1.0f ? 1.0f : (mCurrentIntraDetailLevel<0.0f ? 0.0f : mCurrentIntraDetailLevel);
      return mCurrentDetailLevel;
   }

This replaces everything up until the line "// this function is a little odd"
This is only if you want detail textures to fade away with distance if you only have 1 LOD. If you don't want them fading, I still have the old method posted above.
Edited on Jun 23, 2008 19:06 GMT

Apparatus (Jun 23, 2008 at 03:13 GMT) Resource Rating: 5

Shouldn't detail map work in effect when closer to the object? that's the reason it was ever used: to get the object more detail as you get closer to it; thus I wouldn't worry much if it disappears at a given distance.

Morrock (Jun 23, 2008 at 03:22 GMT)

Well, it does get better detail as you get closer to it with this code. And because it gets more detail the closer you get, obviously when inversed (moving away from it) it will lose detail. The problem is, at about 35 yards, when mCurrentIntraDetailLevel = 0.45 with this object, the detail texture seems to dissappear completely. It's very strange to see a high detail object blend into a moderate detail object (this transition is fine and smooth.) but then very noticeably go from moderate detail suddenly into base-lowest detail; instead of a smooth transition. The thing is, the numbers keep going down and fading, the math is just perfect. But the texture just disappears.
Edited on Jun 23, 2008 03:24 GMT

Apparatus (Jun 23, 2008 at 07:52 GMT) Resource Rating: 5

I am very tired atm but i will try later on again - the clamp issue bothers me a bit - sometimes I get the detail to tile sometimes not; could be the settings in the material but I am not sure (tried in max and milkshape as well); could anyone take a look (max, blender milkshape lightwave - the only exporters I know of can export detailmaps);

Leslie Young (Jun 23, 2008 at 11:17 GMT)

You need to this fulltime (or are you not allready, cause you seem to work really fast). The bridges look really good and this new resource seems to be even better.

Joseph Greenawalt (Jun 23, 2008 at 11:40 GMT)

Quote:
It's very strange to see a high detail object blend into a moderate detail object (this transition is fine and smooth.) but then very noticeably go from moderate detail suddenly into base-lowest detail;

Yeah, it is pretty abrupt. My code was doing the same thing when mCurrentIntraDetailLevel dropped below 0.5. I'm pretty sure it's a problem with the engine, as dts shapes exported from blender with detail maps also do the same thing, even in stock TGE.

@Apparatus:

Quote:
the clamp issue bothers me a bit - sometimes I get the detail to tile sometimes not; could be the settings in the material but I am not sure (tried in max and milkshape as well); could anyone take a look (max, blender milkshape lightwave - the only exporters I know of can export detailmaps);

I think the reason why the test shape that you sent me works with a tiling detail map texture is because the same "cliff00" texture is being used for both the diffuse and the detail map. I'm pretty sure that the SWrap and TWrap flags on the main "cliff00" dts material are carrying over to the detail map rendering pass or texture setup only because they're both using the same texture. Using different textures doesn't work in my tests here; but if I use the same texture for both diffuse and detail map, the detail map tiling works.

Hopefully it's just a matter of setting the right gl texture parameter when setting up the detail texture. I'll see if I can find a fix.
Edited on Jun 23, 2008 13:36 GMT

Joseph Greenawalt (Jun 23, 2008 at 13:51 GMT)

Here's a fix for tiled detail maps. In the TSMesh::resetMaterials() method of tsMesh.cc (somewhere around line 704), add the following two (bold) lines and rebuild:

void TSMesh::resetMaterials()
{
   // restore gl state changes made for dmap
   if (TSShapeInstance::smRenderData.detailMapMethod==TSShapeInstance::DETAIL_MAP_MULTI_1 ||
       TSShapeInstance::smRenderData.detailMapMethod==TSShapeInstance::DETAIL_MAP_MULTI_2)
   {
      // set detail maps texture unit
      glActiveTextureARB(GL_TEXTURE0_ARB + TSShapeInstance::smRenderData.detailMapTE);

      glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL_REPEAT);
      glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_T,GL_REPEAT);

      glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
      glDisable(GL_TEXTURE_2D);

      // may have changed texture matrix of one of detail-texture texture-environment
      if (mFabs(TSShapeInstance::smRenderData.detailTextureScale-1.0f) > 0.001f)
      {
         glMatrixMode(GL_TEXTURE);
         glLoadIdentity();
         glMatrixMode(GL_MODELVIEW);
      }

Edited on Jun 23, 2008 13:53 GMT

Apparatus (Jun 23, 2008 at 15:08 GMT) Resource Rating: 5

It's working like a charm. Perfect!

Morrock (Jun 23, 2008 at 19:09 GMT)

So do you think it would be best if we tweaked the numbers so mCurrentIntraDetailLevel wouldn't drop below 0.45 until at a very far distance away (to where it's barely noticeable) or keep it as is. We could also try finding the engine problem, but I have no idea where to start with that.

edit: Oh, well I found the methods having to do with detail levels.
line 560 void TSShapeInstance::render(const Point3F * objectScale) This gets object details and decides which render function path to take. If it has a detail map and intraLevel != 0 it goes to...

line 633 TSShapeInstance::render(S32 dl, F32 intraDL, const Point3F * objectScale)
Edited on Jun 23, 2008 19:23 GMT

Edward (Jul 11, 2008 at 14:44 GMT)

ok Question, i have this working in the terrains fine. How do i get this to work on a DTS object. Do i simply add a detailmap = #name of detail map. or is there something im missing in the editor to add detail mapping to my dts objects. or is it a exporter option.

Edward (Jul 11, 2008 at 14:48 GMT)

thanks in advance

Joseph Greenawalt (Jul 11, 2008 at 16:18 GMT)

It's an exporter option.

Jason Gossiaux (Sep 14, 2008 at 07:25 GMT)

How do you add detail maps and export them using Milkshape 3D? I wasn't able to find anything on the forums (GGs or theirs) aside from "they work" heh. Thanks!

Edit - I finally figured it out. It appears that in MS3D if my detail map is a jpg or bmp it isn't detected by Showtool. So I'm using PNGs and it is working fine. Adding a detail map for my skin textures really enhanced my models quite a bit. Out of curiosity does detail mapping really provide us with anything? I've read that its just blended with the normal texture, which could have been done on the main texture from the get go. Thanks for explaining this for me.
Edited on Sep 14, 2008 17:41 GMT

Apparatus (Sep 16, 2008 at 17:57 GMT) Resource Rating: 5

Jason, detail maps are useful in particular for large objects, to save color map resolution; instead of using a large 2048x2048 color texture, use a 256x256 or 512x512 (for color information) and a small grayscale detail texture 256x256. The effect is visible when you are up close to the object with this texture combination.

You must be a member and be logged in to either append comments or rate this resource.