Cel Shading in TGE
by DavidRM · 05/29/2006 (10:57 am) · 35 comments
Download Code File
This method of cel shading based on multi-texturing and uses 2 passes. The resulting look, though, seems to be well worth it. And since you can limit which objects get the cel-shading (players, statics, etc) you don't have to take the 2-pass performance hit on everything. Also, I'm only interested in cel shading objects (DTS) not interiors or the terrain. Still, you could probably extrapolate how to cel shade those, as well, if you wanted.
NOTE: The changes described are based on a clean 1.4.
The files in the zip reflect all of these changes. Make sure you put them where they belong in the source tree:
ts\tsMesh.h
ts\tsMesh.cc
ts\tsShapeInstance.h
ts\tsShapeInstance.cc
game\player.cc
ALSO NOTE: If you use the files in the zip, backup your version of those files.
TSMesh.h Changes
Add these changes to TSMesh.h (new code highlighted:
TSMesh.cc Changes
Add these changes to TSMesh.cc (new code highlighted:
TSShapeInstance.h Changes
This one requires editing of existing code (not just adding new stuff).
Change:
To:
OK. Now to "just add new stuff". Code additions highlighted.
TSShapeInstance.cc Changes
Change:
To:
Change:
To:
Change:
To:
And now, for more "just add stuff":
OK. That's it for the core changes. Take a minute and make sure you can get a build. :)
Since the cel shading won't happen without the object transform, and since that parameter is optional (defaulting to NULL), you should be able to build and run and see everything shaded normally by OpenGL.
I went with this approach to make it easy to turn cel shading off and on for particular types of in-game objects. If you want cel shading...just pass in the object transform to the render() call.
For example, to turn on cel shading for the player, in player.cc change:
To:
And change:
To:
And that should do it. Re-build and you'll have cel shading on player models. Even vehicles (though you need to add the object transform to get it on the vehicle wheels).
That's it, I think. What I want to change/extend are:
* Work with interior lights (still waiting to get the lighting kit)
* Have a scriptable, per object cel shading texture
* Better integrate with Torque's multitexturing so I can get it all done in one pass
Known issues:
* Statics with translucent materials (like the trees in the starter.fps) show some artifacts as the cel shading is sometimes appears on top of the billboard thingy.
* Having to call updateSkin() in TSSkinMesh::renderCelShade() is a significant performance hit. You should look into a way to incorporate the cel shading in TSMesh::render() to take advantage of the bone transforms already done.
Comments and critique--and suggestions for improvements--are welcome.
Have fun with it.
-David
Credits
The cel shading articles by Sami "MENTAL" Hamlaoui at Nehe and GameDev were very useful. Specifically:
Lesson 37: Cel-Shading
Cel-Shading
Also, the resourced posted on GG by Alexander "Nanaki" Traverso:
Cel Shading with OpenGL
And I would like to thank Dave "Myopic Rhino" Astle for answering my newb OpenGL questions.
Edits
Edit: I had left out an #include in TSMesh.cc. And I had left a function header out of TSShapeInstance.h (in the zip file, not in the instructions above).
Edit: Needed to disable lighting in TSMesh::render().
Edit: If you have the Torque Lighting Kit installed, you will need to either remove the overexposure from DTS objects as described in this thread (viewable only by TLK licensees) or *don't* disable the lighting in TSMesh::render(). If you use the latter, depending on your ambient light settings, you might see some regular shading on your model, in addition to the cel shading. You'll have to decide which looks better to you and/or which best fits your project.
Edit: Fix for the "Player models sometimes show artifacts of the cel shading of other player models in the scene" problem.
Edit: Forgot the object transform in TSSkinMesh::renderCelShade().
This method of cel shading based on multi-texturing and uses 2 passes. The resulting look, though, seems to be well worth it. And since you can limit which objects get the cel-shading (players, statics, etc) you don't have to take the 2-pass performance hit on everything. Also, I'm only interested in cel shading objects (DTS) not interiors or the terrain. Still, you could probably extrapolate how to cel shade those, as well, if you wanted.
NOTE: The changes described are based on a clean 1.4.
The files in the zip reflect all of these changes. Make sure you put them where they belong in the source tree:
ts\tsMesh.h
ts\tsMesh.cc
ts\tsShapeInstance.h
ts\tsShapeInstance.cc
game\player.cc
ALSO NOTE: If you use the files in the zip, backup your version of those files.
TSMesh.h Changes
Add these changes to TSMesh.h (new code highlighted:
class TSMesh
{
protected:
...
void renderEnvironmentMap(S32 frame, S32 matFrame, TSMaterialList *);
void renderDetailMap(S32 frame, S32 matFrame, TSMaterialList *);
[b][i]virtual void renderCelShade(S32 frame, S32 matFrame, TSMaterialList *, const MatrixF *);[/i][/b]
...
static void initEnvironmentMapMaterials();
static void resetEnvironmentMapMaterials();
static void initDetailMapMaterials();
static void resetDetailMapMaterials();
[b][i]static void initCelShadeMaterials();
static void resetCelShadeMaterials();[/i][/b]
...
};
...
class TSSkinMesh : public TSMesh
{
public:
...
// render methods..
void render(S32 frame, S32 matFrame, TSMaterialList *);
void renderShadow(S32 frame, const MatrixF & mat, S32 dim, U32 * bits, TSMaterialList *);
[b][i]void renderCelShade(S32 frame, S32 matFrame, TSMaterialList *, const MatrixF *);[/i][/b]
...
};TSMesh.cc Changes
Add these changes to TSMesh.cc (new code highlighted:
...
#include "core/frameAllocator.h"
#include "platform/profiler.h"
[b][i]// for cel shading
#include "sceneGraph/sceneGraph.h"[/i][/b]
...
void TSMesh::render(S32 frame, S32 matFrame, TSMaterialList * materials)
{
...
// lock...
bool lockArrays = dglDoesSupportCompiledVertexArray();
if (lockArrays)
glLockArraysEXT(0,vertsPerFrame);
[b][i]// cel shading - begin
bool oldlighting=glIsEnabled(GL_LIGHTING);
glDisable(GL_LIGHTING);
// cel shading - end[/i][/b]
for (S32 i=0; i<primitives.size(); i++)
{
...
}
[b][i]// cel shading - begin
if (oldlighting) glEnable(GL_LIGHTING);
// cel shading - end[/i][/b]
// unlock...
if (lockArrays)
glUnlockArraysEXT();
...
}
...
void TSMesh::resetDetailMapMaterials()
{
...
}
[b][i]//-----------------------------------------------------
// TSMesh render cel shading (2 pass) methods
//-----------------------------------------------------
bool celShadeTextureBuilt=false;
GLuint celShadeTexture;
void buildCelShadeTexture()
{
if (!celShadeTextureBuilt)
{
// build cel shade texture
Vector<Point3F> celShades;
celShades.setSize(32);
for (S32 pp=0; pp<32; pp++)
{
F32 shade=1.0;
if (pp<2)
shade=0.75;
else if (pp<8)
shade=0.80;
celShades[pp]=Point3F(shade,shade,shade);
}
glGenTextures(1,&celShadeTexture);
glBindTexture(GL_TEXTURE_1D,celShadeTexture);
glTexParameteri(GL_TEXTURE_1D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);
glTexParameteri(GL_TEXTURE_1D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);
glTexImage1D(GL_TEXTURE_1D,0,GL_RGB,32,0,GL_RGB,GL_FLOAT,celShades.address());
celShadeTextureBuilt=true;
}
}
void TSMesh::renderCelShade(S32 frame, S32 matFrame, TSMaterialList * materials, const MatrixF *objectTransform)
{
// most gl states assumed to be all set up...
// if we're here, then we're drawing cel shading in two passes...
if ((objectTransform==NULL) || (!dglDoesSupportARBMultitexture()))
return;
S32 firstVert = vertsPerFrame * frame;
const Point3F *normals = getNormals(firstVert);
LightManager * lightManager = gClientSceneGraph->getLightManager();
Point3F lightPos=this->getCenter();
objectTransform->mulP(lightPos);
if (lightManager->getSunLight())
lightPos-=lightManager->getSunLight()->mPos;
else
lightPos=-lightManager->getShadowLightDirection();
VectorF lightVector=lightPos;
MatrixF worldToObj=*objectTransform;
worldToObj.inverse();
worldToObj.mulV(lightVector);
lightVector.normalize();
// set up vertex arrays -- already enabled in TSShapeInstance::render
glVertexPointer(3,GL_FLOAT,0,&verts[firstVert]);
glNormalPointer(GL_FLOAT,0,normals);
// create vertex texture coordinates
static Vector<F32> cstverts;
cstverts.setSize(vertsPerFrame);
for (S32 vv=0; vv<vertsPerFrame; vv++)
{
Point3F tmpNormal=normals[vv];
// dot product of the vertex normal and light
F32 dotP = mDot(tmpNormal,lightVector);
if (dotP<0.0)
dotP=0.0;
cstverts[vv]=dotP;
}
glTexCoordPointer(1,GL_FLOAT,0,cstverts.address());
// lock...
bool lockArrays = dglDoesSupportCompiledVertexArray();
if (lockArrays)
glLockArraysEXT(0,vertsPerFrame);
for (S32 i=0; i<primitives.size(); i++)
{
TSDrawPrimitive & draw = primitives[i];
AssertFatal(draw.matIndex & TSDrawPrimitive::Indexed,"TSMesh::render: rendering of non-indexed meshes no longer supported");
if (!(draw.matIndex & TSDrawPrimitive::NoMaterial))
{
S32 flags = materials->getFlags(draw.matIndex & TSDrawPrimitive::MaterialMask);
// don't draw on translucent material - issues with env mapping
if (flags & TSMaterialList::Translucent)
continue;
}
glDrawElements(getDrawType(draw.matIndex>>30),draw.numElements,GL_UNSIGNED_SHORT,&indices[draw.start]);
}
// unlock...
if (lockArrays)
glUnlockArraysEXT();
}
void TSMesh::initCelShadeMaterials()
{
buildCelShadeTexture();
glEnable(GL_TEXTURE_1D);
glEnable(GL_BLEND);
glEnable(GL_CULL_FACE);
glFrontFace(GL_CW);
glDepthMask(GL_TRUE);
glBlendFunc(GL_ZERO,GL_SRC_COLOR);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_NORMAL_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glBindTexture(GL_TEXTURE_1D,celShadeTexture);
glTexParameteri(GL_TEXTURE_1D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);
glTexParameteri(GL_TEXTURE_1D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);
glDisable(GL_LIGHTING);
}
void TSMesh::resetCelShadeMaterials()
{
glEnable(GL_LIGHTING);
glDisable(GL_BLEND);
glDisable(GL_TEXTURE_1D);
glDepthMask(GL_TRUE);
glDisable(GL_CULL_FACE);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_NORMAL_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
}[/i][/b]
//-----------------------------------------------------
// TSMesh renderShadow
//-----------------------------------------------------
...
void TSSkinMesh::render(S32 frame, S32 matFrame, TSMaterialList * materials)
{
// update verts and normals...
updateSkin();
// render...
Parent::render(frame,matFrame,materials);
}
void TSSkinMesh::renderShadow(S32 frame, const MatrixF & mat, S32 dim, U32 * bits, TSMaterialList * materials)
{
// update verts and normals...
updateSkin();
// render...
Parent::renderShadow(frame,mat,dim,bits,materials);
}
[b][i]void TSSkinMesh::renderCelShade(S32 frame, S32 matFrame, TSMaterialList *materials, const MatrixF *objectTransform)
{
// update verts and normals...
updateSkin();
// render...
Parent::renderCelShade(frame,matFrame,materials,objectTransform);
}[/i][/b]TSShapeInstance.h Changes
This one requires editing of existing code (not just adding new stuff).
Change:
virtual void render(const Point3F * objectScale=NULL); virtual void render(S32 dl, F32 intraDL = 0.0f, const Point3F * objectScale = NULL);
To:
[b][i]virtual void render(const MatrixF *objectTransform=NULL, const Point3F * objectScale=NULL); virtual void render(const MatrixF *objectTransform, S32 dl, F32 intraDL = 0.0f, const Point3F * objectScale = NULL);[/i][/b]
OK. Now to "just add new stuff". Code additions highlighted.
class TSShapeInstance
{
public:
...
/// An ObjectInstance points to the renderable items in the shape...
struct ObjectInstance
{
...
/// Renders the environment map
virtual void renderEnvironmentMap(S32 objectDetail, TSMaterialList *);
/// Renders the detail map
virtual void renderDetailMap(S32 objectDetail, TSMaterialList *);
[b][i]/// Renders the cel shading
virtual void renderCelShade(S32 objectDetail, TSMaterialList *, const MatrixF *);[/i][/b]
/// Renders the fog texture
virtual void renderFog(S32 objectDetail, TSMaterialList*);
...
}
...
/// These are set up by default based on shape data
struct MeshObjectInstance : ObjectInstance
{
...
void renderEnvironmentMap(S32 objectDetail, TSMaterialList *);
void renderDetailMap(S32 objectDetail, TSMaterialList *);
[b][i]void renderCelShade(S32 objectDetail, TSMaterialList *, const MatrixF *);[/i][/b]
void renderShadow(S32 objectDetail, const MatrixF & mat, S32 dim, U32 * bits, TSMaterialList *);
...
}
...
void renderEnvironmentMap();
void renderDetailMap();
[b][i]void renderCelShade(const MatrixF *);[/i][/b]
void renderFog();
...
}TSShapeInstance.cc Changes
Change:
void TSShapeInstance::render(const Point3F * objectScale)
{
...
if (mCurrentIntraDetailLevel>alphaIn+alphaOut)
render(mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
else if (mCurrentIntraDetailLevel>alphaOut)
{
...
// first draw next detail level
if (mCurrentDetailLevel+1<mShape->details.size() && mShape->details[mCurrentDetailLevel+1].size>0.0f)
{
...
render(mCurrentDetailLevel+1,0.0f,objectScale);
}
...
render(mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
}
else
{
...
// first draw next detail level
if (mCurrentDetailLevel+1<mShape->details.size() && mShape->details[mCurrentDetailLevel+1].size>0.0f)
render(mCurrentDetailLevel+1,0.0f,objectScale);
...
render(mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
...
}
...
}To:
void TSShapeInstance::render([b][i]const MatrixF *objectTransform,[/i][/b] const Point3F * objectScale)
{
...
if (mCurrentIntraDetailLevel>alphaIn+alphaOut)
render([b][i]objectTransform,[/i][/b]mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
else if (mCurrentIntraDetailLevel>alphaOut)
{
...
// first draw next detail level
if (mCurrentDetailLevel+1<mShape->details.size() && mShape->details[mCurrentDetailLevel+1].size>0.0f)
{
...
render([b][i]objectTransform,[/i][/b]mCurrentDetailLevel+1,0.0f,objectScale);
}
...
render([b][i]objectTransform,[/i][/b]mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
}
else
{
...
// first draw next detail level
if (mCurrentDetailLevel+1<mShape->details.size() && mShape->details[mCurrentDetailLevel+1].size>0.0f)
render([b][i]objectTransform,[/i][/b]mCurrentDetailLevel+1,0.0f,objectScale);
...
render([b][i]objectTransform,[/i][/b]mCurrentDetailLevel,mCurrentIntraDetailLevel,objectScale);
...
}
...Change:
void TSShapeInstance::render(S32 dl, F32 intraDL, const Point3F * objectScale)
{
...
// render detail maps using a second pass?
if (twoPassDetailMap())
renderDetailMap();
// render environment map using second pass?
if (twoPassEnvironmentMap())
renderEnvironmentMap();
if (!supportBuffers || !renderDecalsX(ss,od))
{
...
}
// render fog if 2-passing it
if (twoPassFog())
renderFog();
...
}To:
void TSShapeInstance::render([b][i]const MatrixF *objectTransform,[/i][/b] S32 dl, F32 intraDL, const Point3F * objectScale)
{
...
// render detail maps using a second pass?
if (twoPassDetailMap())
renderDetailMap();
// render environment map using second pass?
if (twoPassEnvironmentMap())
renderEnvironmentMap();
if (!supportBuffers || !renderDecalsX(ss,od))
{
...
}
[b][i]// drm - render cel shade
if (objectTransform)
renderCelShade(objectTransform);[/i][/b]
// render fog if 2-passing it
if (twoPassFog())
renderFog();
...
}Change:
GBitmap * TSShapeInstance::snapshot(U32 width, U32 height, bool mip, MatrixF & cameraMatrix,bool hiQuality)
{
...
// take a snapshot of the shape with a black background...
glClearColor(0,0,0,0);
glClear(GL_DEPTH_BUFFER_BIT|GL_COLOR_BUFFER_BIT);
GBitmap * blackBmp = new GBitmap;
blackBmp->allocateBitmap(bmpWidth,bmpHeight,false,GBitmap::RGB);
render(mCurrentDetailLevel,mCurrentIntraDetailLevel);
glReadPixels(xcenter-(bmpWidth>>1),ycenter-(bmpHeight>>1),bmpWidth,bmpHeight,GL_RGB,GL_UNSIGNED_BYTE,(void*)blackBmp->getBits(0));
// take a snapshot of the shape with a white background...
glClearColor(1,1,1,1);
glClear(GL_DEPTH_BUFFER_BIT|GL_COLOR_BUFFER_BIT);
GBitmap * whiteBmp = new GBitmap;
whiteBmp->allocateBitmap(bmpWidth,bmpHeight,false,GBitmap::RGB);
render(mCurrentDetailLevel,mCurrentIntraDetailLevel);
glReadPixels(xcenter-(bmpWidth>>1),ycenter-(bmpHeight>>1),bmpWidth,bmpHeight,GL_RGB,GL_UNSIGNED_BYTE,(void*)whiteBmp->getBits(0));
...
}To:
GBitmap * TSShapeInstance::snapshot(U32 width, U32 height, bool mip, MatrixF & cameraMatrix,bool hiQuality)
{
...
// take a snapshot of the shape with a black background...
glClearColor(0,0,0,0);
glClear(GL_DEPTH_BUFFER_BIT|GL_COLOR_BUFFER_BIT);
GBitmap * blackBmp = new GBitmap;
blackBmp->allocateBitmap(bmpWidth,bmpHeight,false,GBitmap::RGB);
render([b][i]NULL,[/i][/b]mCurrentDetailLevel,mCurrentIntraDetailLevel);
glReadPixels(xcenter-(bmpWidth>>1),ycenter-(bmpHeight>>1),bmpWidth,bmpHeight,GL_RGB,GL_UNSIGNED_BYTE,(void*)blackBmp->getBits(0));
// take a snapshot of the shape with a white background...
glClearColor(1,1,1,1);
glClear(GL_DEPTH_BUFFER_BIT|GL_COLOR_BUFFER_BIT);
GBitmap * whiteBmp = new GBitmap;
whiteBmp->allocateBitmap(bmpWidth,bmpHeight,false,GBitmap::RGB);
render([b][i]NULL,[/i][/b]mCurrentDetailLevel,mCurrentIntraDetailLevel);
glReadPixels(xcenter-(bmpWidth>>1),ycenter-(bmpHeight>>1),bmpWidth,bmpHeight,GL_RGB,GL_UNSIGNED_BYTE,(void*)whiteBmp->getBits(0));
...
}And now, for more "just add stuff":
void TSShapeInstance::renderDetailMap()
{
...
}
[b][i]void TSShapeInstance::renderCelShade(const MatrixF *objectTransform)
{
S32 dl = smRenderData.detailLevel;
const TSDetail * detail = &mShape->details[dl];
S32 ss = detail->subShapeNum;
S32 od = detail->objectDetailNum;
S32 start = mShape->subShapeFirstObject[ss];
S32 end = mShape->subShapeNumObjects[ss] + start;
// set up gl environment for the cel shading
TSMesh::initCelShadeMaterials();
// run through objects and render cel shading
smRenderData.currentTransform = NULL;
for (S32 i=start; i<end; i++)
mMeshObjects[i].renderCelShade(od,mMaterialList,objectTransform);
// if we have a matrix pushed, pop it now
if (smRenderData.currentTransform)
glPopMatrix();
// restore gl state
TSMesh::resetCelShadeMaterials();
}[/i][/b]
S32 TSShapeInstance::getCurrentDetail()
{
return mCurrentDetailLevel;
}
...
void TSShapeInstance::ObjectInstance::renderDetailMap(S32, TSMaterialList *)
{
...
}
[b][i]void TSShapeInstance::ObjectInstance::renderCelShade(S32, TSMaterialList *, const MatrixF *)
{
AssertFatal(0,"TSShapeInstance::ObjectInstance::renderCelShade: no default render method.");
}[/i][/b]
void TSShapeInstance::ObjectInstance::renderFog(S32, TSMaterialList*)
{
...
}
...
void TSShapeInstance::MeshObjectInstance::renderDetailMap(S32 objectDetail, TSMaterialList * materials)
{
...
}
[b][i]void TSShapeInstance::MeshObjectInstance::renderCelShade(S32 objectDetail, TSMaterialList * materials, const MatrixF *objectTransform)
{
if (visible>0.01f)
{
TSMesh * mesh = getMesh(objectDetail);
if (mesh)
{
MatrixF * transform = getTransform();
if (transform != TSShapeInstance::smRenderData.currentTransform)
{
if (TSShapeInstance::smRenderData.currentTransform)
glPopMatrix();
if (transform)
{
glPushMatrix();
dglMultMatrix(transform);
}
TSShapeInstance::smRenderData.currentTransform = transform;
}
if (TSShapeInstance::smRenderData.balloonShape)
{
glPushMatrix();
F32 & bv = TSShapeInstance::smRenderData.balloonValue;
glScalef(bv,bv,bv);
}
if (objectTransform)
{
MatrixF mat=*objectTransform;
if (transform)
mat.mul(*transform);
mesh->renderCelShade(frame,matFrame,materials,&mat);
}
else
mesh->renderCelShade(frame,matFrame,materials,NULL);
if (TSShapeInstance::smRenderData.balloonShape)
glPopMatrix();
}
}
}[/i][/b]
void TSShapeInstance::MeshObjectInstance::renderShadow(S32 objectDetail, const MatrixF & mat, S32 dim, U32 * bits, TSMaterialList * materialList)
{
...
}OK. That's it for the core changes. Take a minute and make sure you can get a build. :)
Since the cel shading won't happen without the object transform, and since that parameter is optional (defaulting to NULL), you should be able to build and run and see everything shaded normally by OpenGL.
I went with this approach to make it easy to turn cel shading off and on for particular types of in-game objects. If you want cel shading...just pass in the object transform to the render() call.
For example, to turn on cel shading for the player, in player.cc change:
...
void Player::renderMountedImage(SceneState* state, ShapeImageRenderImage* rimage)
{
...
image.shapeInstance->setupFog(fogAmount,state->getFogColor());
image.shapeInstance->animate();
image.shapeInstance->render();
...
}
...To:
...
void Player::renderMountedImage(SceneState* state, ShapeImageRenderImage* rimage)
{
...
image.shapeInstance->setupFog(fogAmount,state->getFogColor());
image.shapeInstance->animate();
[b][i]// local variable mat (MatrixF) holds the correct transform
image.shapeInstance->render(&mat);[/i][/b]
...
}
...And change:
...
void Player::renderImage(SceneState* state, SceneRenderImage* image)
{
...
TSMesh::setOverrideFade( mFadeVal );
mShapeInstance->setupFog(fogAmount,state->getFogColor());
mShapeInstance->animate();
mShapeInstance->render();
TSMesh::setOverrideFade( 1.0 );
...
}To:
...
void Player::renderImage(SceneState* state, SceneRenderImage* image)
{
...
TSMesh::setOverrideFade( mFadeVal );
mShapeInstance->setupFog(fogAmount,state->getFogColor());
mShapeInstance->animate();
[b][i]MatrixF mat=getRenderTransform();
mShapeInstance->render(&mat);[/i][/b]
TSMesh::setOverrideFade( 1.0 );
...
}And that should do it. Re-build and you'll have cel shading on player models. Even vehicles (though you need to add the object transform to get it on the vehicle wheels).
That's it, I think. What I want to change/extend are:
* Work with interior lights (still waiting to get the lighting kit)
* Have a scriptable, per object cel shading texture
* Better integrate with Torque's multitexturing so I can get it all done in one pass
Known issues:
* Statics with translucent materials (like the trees in the starter.fps) show some artifacts as the cel shading is sometimes appears on top of the billboard thingy.
* Having to call updateSkin() in TSSkinMesh::renderCelShade() is a significant performance hit. You should look into a way to incorporate the cel shading in TSMesh::render() to take advantage of the bone transforms already done.
Comments and critique--and suggestions for improvements--are welcome.
Have fun with it.
-David
Credits
The cel shading articles by Sami "MENTAL" Hamlaoui at Nehe and GameDev were very useful. Specifically:
Lesson 37: Cel-Shading
Cel-Shading
Also, the resourced posted on GG by Alexander "Nanaki" Traverso:
Cel Shading with OpenGL
And I would like to thank Dave "Myopic Rhino" Astle for answering my newb OpenGL questions.
Edits
Edit: I had left out an #include in TSMesh.cc. And I had left a function header out of TSShapeInstance.h (in the zip file, not in the instructions above).
Edit: Needed to disable lighting in TSMesh::render().
Edit: If you have the Torque Lighting Kit installed, you will need to either remove the overexposure from DTS objects as described in this thread (viewable only by TLK licensees) or *don't* disable the lighting in TSMesh::render(). If you use the latter, depending on your ambient light settings, you might see some regular shading on your model, in addition to the cel shading. You'll have to decide which looks better to you and/or which best fits your project.
Edit: Fix for the "Player models sometimes show artifacts of the cel shading of other player models in the scene" problem.
Edit: Forgot the object transform in TSSkinMesh::renderCelShade().
About the author
#22
Edit: Ok, i got starter.fps to apply that cel shading texture, looks great. But how come all my static objects are now blown out without any shading at all? I have simple textures right now (one color) and basically they turned into one color blob.
Edit: Alright I'm an idiot and didn't read the manual. Sorry :( I applied these to my static objects in tsStatic.cc, oh man it's sooooo awesome!
06/26/2006 (6:23 pm)
Anyone have any ideas why my models aren't being cel shaded? They just look like blown out over exposed models now, I notice everything is much brighter than usual, but as far as shading on the model itself, I don't see any at all.Edit: Ok, i got starter.fps to apply that cel shading texture, looks great. But how come all my static objects are now blown out without any shading at all? I have simple textures right now (one color) and basically they turned into one color blob.
Edit: Alright I'm an idiot and didn't read the manual. Sorry :( I applied these to my static objects in tsStatic.cc, oh man it's sooooo awesome!
#23
BTW There's a very interesting effect when keeping lighting disabled and cel shading is applied.
06/26/2006 (7:39 pm)
I noticed that because of the lighting disable in tsmesh that regular lighting is disabled for non cel shaded objects. Any pointers on what how to go about distinguishing between cel and regular shading?BTW There's a very interesting effect when keeping lighting disabled and cel shading is applied.
#24
I've been trying to make some edits to it and it seems there's a small bug. When the mission loads, the shadows seem to be in the right position with respect to the direction of the sun. However, if I go into the Mission Editor and move the sun, the shadows don't move with it. For example, if I change the sun's azimuth by 180 degrees, thus moving it to the opposite side, the cel rendered shadows will actually be facing the sun since it appears to be based on the initial sun position.
Any ideas on how to fix this?
07/17/2006 (2:16 pm)
Great Resource!I've been trying to make some edits to it and it seems there's a small bug. When the mission loads, the shadows seem to be in the right position with respect to the direction of the sun. However, if I go into the Mission Editor and move the sun, the shadows don't move with it. For example, if I change the sun's azimuth by 180 degrees, thus moving it to the opposite side, the cel rendered shadows will actually be facing the sun since it appears to be based on the initial sun position.
Any ideas on how to fix this?
#27
Are you updating the sun? Or fxSunlight? Because the cel shading doesn't track fxSunlight.
If you save and reload the mission, does it reflect the changed direction?
-David
07/17/2006 (2:54 pm)
Not really sure what to suggest. The code uses the light manager to get the sun, every frame for every object.Are you updating the sun? Or fxSunlight? Because the cel shading doesn't track fxSunlight.
If you save and reload the mission, does it reflect the changed direction?
-David
#28
Right, that's why i'm all confused about it. I think its the actual sun, not fxSunlight: if I relight the scene after moving the sun, all the other shadows move respectively, its just the cel shading that stays the same.
However, if I save and reload it, the shadow then moves to a new direction (I don't believe its either the direction of the original sun or the 180 degree rotated sun). That makes things interesting...
07/17/2006 (3:19 pm)
@David: Right, that's why i'm all confused about it. I think its the actual sun, not fxSunlight: if I relight the scene after moving the sun, all the other shadows move respectively, its just the cel shading that stays the same.
However, if I save and reload it, the shadow then moves to a new direction (I don't believe its either the direction of the original sun or the 180 degree rotated sun). That makes things interesting...
#29
Inside the funtion TSMesh::renderCelShade(), instead of
Replace it with:
Edit: Layout
07/18/2006 (8:45 am)
I played around with it a bit more, and I've found a fix that seems to solve the problem.Inside the funtion TSMesh::renderCelShade(), instead of
Point3F lightPos=this->getCenter();
objectTransform->mulP(lightPos);
if (lightManager->getSunLight())
lightPos-=lightManager->getSunLight()->mPos;
else
lightPos=-lightManager->getShadowLightDirection();
VectorF lightVector=lightPos;
MatrixF worldToObj=*objectTransform;
worldToObj.inverse();
worldToObj.mulV(lightVector);
lightVector.normalize();Replace it with:
VectorF lightVector = -lightManager->getSunLight()->mDirection; MatrixF worldToObj=*objectTransform; worldToObj.inverse(); worldToObj.mulV(lightVector); lightVector.normalize();
Edit: Layout
#30
Awesome resource. I just got started with Torque not to long ago (I've owned an indie license for a couple years now but never really got started) and always wanted to do a game with cel shading. This is just what I needed to get going down the right path! =)
Thanks!
One question though, I'm wanting to apply this to terrain and interiors as well, however when I'm looking at the interiors (havn't quite gotten to looking at the source for terrain rendering) it looks like it's done alot different than the player models or shapes. I figured it would be similair...
I'm still sharpening my teeth with C++, I've developed with a lot of different interpreted languages and figured it was time to start digging into the engine itself and see what I could break!
Any pointers on how this would get implemented or what I should be looking for when it comes to terrains or interiors would be appreciated. =)
12/27/2006 (1:35 pm)
Hello,Awesome resource. I just got started with Torque not to long ago (I've owned an indie license for a couple years now but never really got started) and always wanted to do a game with cel shading. This is just what I needed to get going down the right path! =)
Thanks!
One question though, I'm wanting to apply this to terrain and interiors as well, however when I'm looking at the interiors (havn't quite gotten to looking at the source for terrain rendering) it looks like it's done alot different than the player models or shapes. I figured it would be similair...
I'm still sharpening my teeth with C++, I've developed with a lot of different interpreted languages and figured it was time to start digging into the engine itself and see what I could break!
Any pointers on how this would get implemented or what I should be looking for when it comes to terrains or interiors would be appreciated. =)
#31
I never looked into implementing cel shading on terrain and interiors so I haven't any tips to offer.
Except...the primitive rendering loop is roughly similar in all 3 cases. What you could do is find the rendering code for interiors and terrain, and compare it with rendering DTS objects. That should give you some clues about where you would need to make changes.
Keep in mind, though, that doing a two-pass cel shade render on every polygon on the screen will add up quickly. That's one reason we decided to limit the cel shading to DTS objects (the other, primary reason was artistic).
Best of luck!
-David
12/27/2006 (6:30 pm)
Jared,I never looked into implementing cel shading on terrain and interiors so I haven't any tips to offer.
Except...the primitive rendering loop is roughly similar in all 3 cases. What you could do is find the rendering code for interiors and terrain, and compare it with rendering DTS objects. That should give you some clues about where you would need to make changes.
Keep in mind, though, that doing a two-pass cel shade render on every polygon on the screen will add up quickly. That's one reason we decided to limit the cel shading to DTS objects (the other, primary reason was artistic).
Best of luck!
-David
#32
Thanks for the advice. I certainly don't want to crush the FPS. The objective is certainly not to create a game requiring top end graphics in order to play!
Just something visually pleasing. I found this link in the other commentary, and kind of did some poking around: http://tdn.garagegames.com/wiki/WorldBuilding/Visual_Style/Cel_Shading/Code
Basically following the instructions there, they do all their work in void Interior::render.
I tested it out, just on the interiors, and noticed it is a major FPS hit, just having one interior in the scene is pretty GPU intensive.
=(
I have a hunch your method would work better, however the way the interiors render compared to the DTS objects is kind of confusing. I'm not even sure where to start!
I've decided to use just your outlining resource (http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=10368) instead of full blown cel shading on the interior models, I can always use simple textures and possibly make some various lighting (and other visual) tweaks to get something that looks how I want.
The Interior::render
looks similair to the TSMesh.cc
So I assume I should be able to munge around and get something to work eventually. My major problem is the fact that I can't find anything remotley like the TSMesh class declaration or the TSSkinMesh class declaration in any of the interior header files... = /
I'm gonna go out on a limb here and assume I would need to reproduce something like ts\custom\tsMeshOutline.h and ts\custom\tsMeshOutline.cc only specifically for the interiors in order to get this to work. In other words I highly doubt that I can re-use it verbatim.
When I get this working I'll post it as a resource and throw a link to your resource in it. It's probably going to take me a while though. =P
12/28/2006 (2:28 pm)
Hey David,Thanks for the advice. I certainly don't want to crush the FPS. The objective is certainly not to create a game requiring top end graphics in order to play!
Just something visually pleasing. I found this link in the other commentary, and kind of did some poking around: http://tdn.garagegames.com/wiki/WorldBuilding/Visual_Style/Cel_Shading/Code
Basically following the instructions there, they do all their work in void Interior::render.
I tested it out, just on the interiors, and noticed it is a major FPS hit, just having one interior in the scene is pretty GPU intensive.
=(
I have a hunch your method would work better, however the way the interiors render compared to the DTS objects is kind of confusing. I'm not even sure where to start!
I've decided to use just your outlining resource (http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=10368) instead of full blown cel shading on the interior models, I can always use simple textures and possibly make some various lighting (and other visual) tweaks to get something that looks how I want.
The Interior::render
void Interior::render(const bool useAlarmLighting, MaterialList* pMaterials, const LM_HANDLE instanceHandle,
const Vector<ColorI>* normalVLights,
const Vector<ColorI>* alarmVLights)looks similair to the TSMesh.cc
// shape outline resource void TSMesh::render(S32 frame, S32 matFrame, TSMaterialList * materials, const MatrixF *objectTransform) // shape outline resource
So I assume I should be able to munge around and get something to work eventually. My major problem is the fact that I can't find anything remotley like the TSMesh class declaration or the TSSkinMesh class declaration in any of the interior header files... = /
I'm gonna go out on a limb here and assume I would need to reproduce something like ts\custom\tsMeshOutline.h and ts\custom\tsMeshOutline.cc only specifically for the interiors in order to get this to work. In other words I highly doubt that I can re-use it verbatim.
When I get this working I'll post it as a resource and throw a link to your resource in it. It's probably going to take me a while though. =P
#33
(I tried with Daniel Ly fix)
10/07/2008 (11:13 pm)
It gives me some issues with TGB 1.5.2 :13>..\engine\ts\tsShapeInstance.cc(617) : error C2511: 'void TSShapeInstance::render(const Point3F *)' : overloaded member function not found in 'TSShapeInstance' 13> ../engine\ts/tsShapeInstance.h(75) : see declaration of 'TSShapeInstance' 13>tsMesh.cc 13>..\engine\ts\tsMesh.cc(1465) : error C2039: 'getSunLight()' : is not a member of 'LightManager' 13> ../engine\lightingSystem/sgLightManager.h(103) : see declaration of 'LightManager' 13>..\engine\ts\tsMesh.cc(1465) : error C2227: left of '->mDirection' must point to class/struct/union/generic typeAny idea on how to make it work?
(I tried with Daniel Ly fix)
#34
Warning 1 warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/INCREMENTAL:NO' specification GLU2D3D.obj
Error 2 error C2065: 'tDiffuse' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 57
Error 3 error C3861: 'getUVs': identifier not found c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 57
Error 4 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 5 error C2039: 'LIGHT_MAP_MULTI' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 6 error C2065: 'LIGHT_MAP_MULTI' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 7 error C2065: 'tLightmap' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 69
Error 8 error C3861: 'getUVs': identifier not found c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 69
Error 9 error C2039: 'lightMapTE' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 70
Error 10 error C2039: 'getUVs' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 11 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 12 error C2065: 'tDiffuse' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 13 error C2039: 'getUVs' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 915
Error 14 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 915
Error 15 error C2039: 'getUVs' : is not a member of 'TSSkinMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 1025
Error 16 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 1025
Error 17 error C2039: 'twoPassLightMap' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 227
Error 18 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 19 error C2039: 'LIGHT_MAP_TWO_PASS' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 20 error C2065: 'LIGHT_MAP_TWO_PASS' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 21 error C2039: 'initLightMapMaterials' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 330
Error 22 error C3861: 'initLightMapMaterials': identifier not found c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 330
Error 23 error C2039: 'renderLightMap' : is not a member of 'TSShapeInstance::MeshObjectInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 335
Error 24 error C2039: 'resetLightMapMaterials' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 342
Error 25 error C3861: 'resetLightMapMaterials': identifier not found c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 342
Error 26 error C2039: 'getSunLight' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1358
Error 27 error C2039: 'getSunLight' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1359
Error 28 error C2227: left of '->mPos' must point to class/struct/union/generic type c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1359
Error 29 error C2039: 'getShadowLightDirection' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1361
Error 30 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 31 error C2039: 'NO_LIGHT_MAP' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 32 error C2065: 'NO_LIGHT_MAP' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 33 error C2065: 'mAnchorPoint' : undeclared identifier c:\torque\tge_1_5_2\engine\game\player.cc 751
Error 34 error C2065: 'mGenerateShadow' : undeclared identifier c:\torque\tge_1_5_2\engine\game\player.cc 783
Error 35 error BK1506 : cannot open file '..\tools\out.vc8.debug_fast\interiorCollision.sbr': No such file or directory BSCMAKE
any idea why or how to fix?
01/20/2009 (8:11 pm)
When i tried to recompile it using TGE 1.5.2 it has some issues. Warning 1 warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/INCREMENTAL:NO' specification GLU2D3D.obj
Error 2 error C2065: 'tDiffuse' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 57
Error 3 error C3861: 'getUVs': identifier not found c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 57
Error 4 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 5 error C2039: 'LIGHT_MAP_MULTI' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 6 error C2065: 'LIGHT_MAP_MULTI' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 66
Error 7 error C2065: 'tLightmap' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 69
Error 8 error C3861: 'getUVs': identifier not found c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 69
Error 9 error C2039: 'lightMapTE' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tssortedmesh.cc 70
Error 10 error C2039: 'getUVs' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 11 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 12 error C2065: 'tDiffuse' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tsshape.cc 871
Error 13 error C2039: 'getUVs' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 915
Error 14 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 915
Error 15 error C2039: 'getUVs' : is not a member of 'TSSkinMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 1025
Error 16 error C2039: 'tDiffuse' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tsshape.cc 1025
Error 17 error C2039: 'twoPassLightMap' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 227
Error 18 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 19 error C2039: 'LIGHT_MAP_TWO_PASS' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 20 error C2065: 'LIGHT_MAP_TWO_PASS' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 327
Error 21 error C2039: 'initLightMapMaterials' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 330
Error 22 error C3861: 'initLightMapMaterials': identifier not found c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 330
Error 23 error C2039: 'renderLightMap' : is not a member of 'TSShapeInstance::MeshObjectInstance' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 335
Error 24 error C2039: 'resetLightMapMaterials' : is not a member of 'TSMesh' c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 342
Error 25 error C3861: 'resetLightMapMaterials': identifier not found c:\torque\tge_1_5_2\engine\ts\tspartinstance.cc 342
Error 26 error C2039: 'getSunLight' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1358
Error 27 error C2039: 'getSunLight' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1359
Error 28 error C2227: left of '->mPos' must point to class/struct/union/generic type c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1359
Error 29 error C2039: 'getShadowLightDirection' : is not a member of 'LightManager' c:\torque\tge_1_5_2\engine\ts\tsmesh.cc 1361
Error 30 error C2039: 'lightMapMethod' : is not a member of 'TSShapeInstance::RenderData' c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 31 error C2039: 'NO_LIGHT_MAP' : is not a member of 'TSShapeInstance' c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 32 error C2065: 'NO_LIGHT_MAP' : undeclared identifier c:\torque\tge_1_5_2\engine\ts\tsdecal.cc 97
Error 33 error C2065: 'mAnchorPoint' : undeclared identifier c:\torque\tge_1_5_2\engine\game\player.cc 751
Error 34 error C2065: 'mGenerateShadow' : undeclared identifier c:\torque\tge_1_5_2\engine\game\player.cc 783
Error 35 error BK1506 : cannot open file '..\tools\out.vc8.debug_fast\interiorCollision.sbr': No such file or directory BSCMAKE
any idea why or how to fix?
#35
09/09/2009 (5:39 am)
I had no troubles implementing this into TGE 1.5.2, use WinMerge to compare changes between the TGE 1.4.2 files and TGE 1.5.2 files 
Torque 3D Owner Tom Perry