Per-Pixel Lighting and Shadowing

#1 Doesn't using DOT3 Bump mapping actually give you "per-pixel" lighting for 'free'? Torque has shadowing, just not projected / self shadowing etc. It's something that would probably be fairly ineffective for exteriors anyway.

Anyway both effects can be achieved by the various shader technologies and they'll no doubt end up in Torque one day ;)

Great link though, thanks.

#2 I see the main use of these techniques for interiors anyway. I even think it would be beneficial to NOT use it for the terrain and stuff, only for interiors. Those however would look MUCH much better. You'd basically get the looks of Doom3 :)

#3 check out some exterior game shots that use all that so called un-needed fancy stuff!

www.stalker-game.com/

matter of fact ALL the exterior shots look awesome, so I think you code it they will use it!

#4 Why oh why do people write website that re-size your browser!? Everytime I move my mouse over that website's menu, it resizes Phoenix! Stop it already!

#5 As Josef implied in his first post (and I completly missed) he was refering to mainly interiors (Bump mapping is of course great for characters as well). Stalker looks pretty darn good, and makes excellent use of foilage.

However the main character looks self-shadowed which is something we were on about...

I don't think we need these features for our games, but I do think it's what the next gen of games will require.

#6 If noone else is doing it, I'll probably try to just throw out the lightmap pass on interior rendering, and insert per-pixel lighting and specular highlights. I don't know if I can tackle shadows though - it looks kinda complicated to me.

EDIT: Yes, Bumpmapping requires building a 'normal map', which is also needed for stuff like specular highlights. I'm not too interested in Bumpmapping myself, though if things go well I'll probably add it anyway.

Damn, I wish we had full CG integration in Torque ;)

#7 >Mike

Isn't Phoenix mozilla based? You can prevent scripts resizing windows.

#8 @Robin: Yes Phoenix does use the Mozilla rendering engine. Maybe I'll check the Mozilla site for info on how to stop the resizing.

#9 I have to admit that so far, I've failed miserably. I get lots of crap, I figure I'm doing something wrong with the normal maps. It really frustrates me, even more so because Quake 1 looks absolutely stunning using TenebraeQuake (http://tenebrae.sourceforge.net)

I really think we should coordinate our effort in that direction. Having a solid engine is a good thing, but Torque needs to be polished up to remain on-par with the latest development. If that doesn't convince you, imagine a picture on bluesnews showing Doom3-like graphics with the headline "...and you can get it for just $100". IMHO we *do* need those features.

#10 We do? Damn, my old GeForce 2 MX 400 would swollow, gag, and vomit that stalkers game if I tried to play it. :P

My point is that by including such graphically shocking features, you're limiting your potential audience. IMO, Torque is good enough to make good looking games. Enough said, now get back to work. :)

#11 You got something wrong here. You don't have to limit Torque to this features. It's perfectly possible to keep the old code in place, and make these features toggleable. So people with a decent systems see them, whereas folks with older card can still play smoothly with the current level of eyecandy.

I'm certain that you'll be crying for more eyecandy the day you buy a new gfx card. Suddenly you'll say "Well I have a good enough card, so why doesn't Torque support this stuff? Doom3 does!" ;)

#12 I agree with you Josef. I'm currently working on including shader support for the TGE using CG. Maybe we can exchange some valuable information about it.

If we can develop a solid framework for shaders in the TGE then we can have bump mapping, real-time reflections, hardware processed bone animation and much more, without making the engine useless for old hardware.

Drop me a note if you're interested. Thanx.

#13 The problem with per-pixel effects is that it requires recent hardware to do it - GeForce 3+ or ATi equivalent. Even then, the fragment shaders on a GeForce4 doesn't allow you to do very many things for lighting and shadowing. You really need the full programmability of an FX to get what you are describing. I've tinkered around in Cg, and I have a good feel of its limitations.

Just to do per pixel normal map reflections, you need 4 TMUs for passing texture coordinate information. It's just not very practical right now.

Though, Cg can easily be worked into the engine. It only took me a couple of hours to get it working on water, and you can see the results: hobbiticus.acm.jhu.edu/stuff/legends_0007.png. Note that that is not nearly finished - it's just per-pixel reflection over a cube map. It seems like new and exciting technology with no limits, but it has some serious limitations right now. It would be worth learning it now and applying it a few years down the line when shaders become standardized and widely supported. Now just isn't the time for shaders in torque for the wide majority of games.

EDIT: helps to link to the right screeny

#14 That's not true, Chris. If you're willing to multipass, you can do per pixel lighting (with normalmaps) on any board that's got the dot3 extension and two TMUs.

You can do quite credible bumpmapping on a Geforce 2 - no fragment programs/pixel shaders are required.

#15 Yeah, I'm not argueing about bump mapping. You can do what people call "dot3 bump mapping" which is really just per-pixel lighting using 2 TMUs. Though, the way it's done using a normalization cube map which could argueable be considered a hack because the rendering pipeline doesn't support normalization of vectors, so you have to give it a normalization cube map for it to normalize its vectors.

Plus, multipass is not usually something you want to do especially if there is transparency and lighting together involved. I know for a fact that you can't do dot3 bump mapping with transparency, but you could using a fragment program that supports such things, which would be GeForce3+ or equivalent.

If you have enough TMUs, you can do transparency and per-pixel lighting in one pass, but to get your surface looking really good, you'd probably want a little bit of reflection. All of that adds up to 6 TMUs for a single pass, possibly more for other effects you'd want. (1 for normal map, 3 for a varying matrix, 1 normalization cube map, 1 base texture) 2 TMU cards would have to do it in 3 passes, which is pretty darn slow (but, that's saying your 2 TMU card supports proper vertex/fragment program profiles). Even a GeForce 3/4 would have to double pass.

#16

Quote:
Yeah, I'm not argueing about bump mapping. You can do what people call "dot3 bump mapping" which is really just per-pixel lighting using 2 TMUs. Though, the way it's done using a normalization cube map which could argueable be considered a hack because the rendering pipeline doesn't support normalization of vectors, so you have to give it a normalization cube map for it to normalize its vectors.

Well, even with current generation fragment programs you need to do this. It wasn't until the newest Radeon/Geforce FX boards that you could normalize the vectors with math, and even so the performance is pretty much the same versus a normalization map.

Quote:
Plus, multipass is not usually something you want to do especially if there is transparency and lighting together involved.

Any dynamic lighting program, fragment programs or not, has to multipass to get an arbitrary number of lights. Doom 3 will definately multipass for multiple lights. Heck, Aquanox will even multipass with just _one_ light. It's better to avoid when at all possible, but if your game is CPU limited anyway (and most are) then just pour that geometry on if you really have a reason to.

Quote:
I know for a fact that you can't do dot3 bump mapping with transparency, but you could using a fragment program that supports such things, which would be GeForce3+ or equivalent.

Hmm. You're sorta right about that, because you need to use the destination alpha to hold lighting info when you're doing per pixel stuff on the older hardware. In practice, you only have an issue when you're 'stacking' multiple translucent objects, I'm pretty sure. EDIT: Duh. What was I smoking? translucent objects usually don't look at the destination alpha. Since I was off in left field somewhere, what transparency problem were you referring to, Chris?

Quote:
If you have enough TMUs, you can do transparency and per-pixel lighting in one pass, but to get your surface looking really good, you'd probably want a little bit of reflection. All of that adds up to 6 TMUs for a single pass, possibly more for other effects you'd want. (1 for normal map, 3 for a varying matrix, 1 normalization cube map, 1 base texture) 2 TMU cards would have to do it in 3 passes, which is pretty darn slow (but, that's saying your 2 TMU card supports proper vertex/fragment program profiles). Even a GeForce 3/4 would have to double pass.

So you turn off environment mapping on older hardware. It's not a bad thing to scale graphical features with the hardware. Shaders are a great idea for torque, you just have to provide fallbacks when hardware support isn't present.

Even without reflection, my 'best' bumpmapping uses TMUs for normalization, normalmap, glossmap, attentuation maps, etc, etc. But you don't necessarily have to do all of that on a GF2 just because you do it on a Geforce FX. (though you can, and it mostly works fine, if your players don't mind getting 10 FPS or whatever.)

#17 It is very easy to implement per-pixel lighting on pre-GF3 class hardware, it all depends on your lighting equation. First off you don't need to rescale the normal with a cube map on hardware that would suffer with the added pass. Ya the visual quality is a little lower, but compared to an engine with only diffuse lighting I'd call it an improvement.

As for reflections that's easy. For smooth surfaces you have reflection mapping. For bumpy surfaces on new hardware you lookup into a cube map per fragment and for old hardware you use the old specular fallback (half vector in tangent space, looks good and does the job).

As for bumped transparencies, rearrange your passes, trust me its possible.

You can easily get per-pixel lighting with bump, diffuse, and specular into 3 passes on the GF2 (2tus), 4 for transparencies. Keep in mind that with a shader system not all objects will require all of the work, some may only be one pass.


Another concern voiced was fill rate. My GF2 Pro smokes, you would be surprised what you can do with it, but I'm unfamiliar with the GF2 MX and last I checked the most recent HL hardware poll still showed GF2 series as the most common, so this could be an issue.

John.

#18 Have you ever tried to do bumped transparencies using an alpha map and the dot3 extension? It is not very possible. It sorta works, but not well at all.

#19 Yes I have.

Look, working from the example I gave (diffuse, bump, ignore specular for now), here is my lighting equation:

Lattn = lighting attenuation.
Li = lighting intensity.
Lc = light color.
NM = normal map.
LV = lighting vector in tangent space.
D = diffuse.
SRC = blend function source.
DST = blend function destination.

Solid = (Lattn - Li) * Lc * (dot3(NM, LV)) * D.RGB
Alpha = (Lattn - Li) * Lc * (dot3(NM, LV)) * D.RGB * D.A + (1 - D.A) * DST.RGB;

------------------------------------------------

This creates the colored light:
(Lattn - Li) * Lc

This does the bump:
(dot3(NM, LV))

This is the diffuse texture:
D.RGB

And this is the blend function set to (SRC_ALPHA, ONE_MINUS_SRC_ALPHA):
D.A + (1 - D.A) * DST.RGB

So far this should make sense.

------------------------------------------------

Now like I said to make this work you need to rearrange some things, but first we do the normal 'breakup into passes':

These will result in scalars (can go into DST.A):
(Lattn - Li)
and
(dot3(NM, LV))
so do these first.

So an un-optimized pass structure would look like this:
Pass1 = (Lattn - Li) -> BlendFunc(ONE, ZERO) writing only to alpha
Pass2 = (dot3(NM, LV)) -> BlendFunc(DST.A, ZERO) writing only to alpha
Pass3 = Lc * D.RGB -> BlendFunc(DST.A, ZERO) writing to rgb

------------------------------------------------

Ok now for the alpha right? The problem is that you are trying to squeeze BlendFunc(SRC_ALPHA, ONE_MINUS_SRC_ALPHA) into the multi-pass structure. Do not use this blend func. Remember you're only trying to achieve the same equation it doesn't need to be in the same order.

Start with the scalars:
Pass1 = (Lattn - Li) -> BlendFunc(ONE, ZERO) writing only to alpha
Pass2 = (dot3(NM, LV)) -> BlendFunc(DST.A, ZERO) writing only to alpha

Now do the (SRC * SRC_ALPHA) in the TU:
Pass3 = Lc * D.RGB * D.A

And change the blend function to:
BlendFunc(DST.A, ONE_MINUS_SRC_ALPHA) writing to rgb

See? You still managed to multiply the lighting, by the diffuse, by the SRC.A and add that to the result of the DST multiplied by one minus the SRC.A.

Also keep in mind that if you can't get your lighting equation working (for example to many vectors, i.e. nowhere to temp store values) you can always fudge it. It doesn't have to be right; it just needs to look good. :)

------------------------------------------------

Ok specular this is the easiest part. You should only be adding to the frame buffer for specular anyway, so just modulate fragment with the diffuse alpha before adding. Specular and reflections really need to be handled on a material level. For instance lets say you had glass, the alpha wouldn't necessarily affect the reflectivity, however if you are using the alpha as a mask is would.

------------------------------------------------

I hope this make sense. If you need me to clarify anything let me know...

John.

#20 Well, I guess it works, but I just don't have the time or energy to go about implementing that. It really sucks trying to do this sort of stuff without knowing linear algebra :\