Game Development Community

Supporting Double-Byte character

by Samme Ng · in Torque Game Engine · 12/22/2003 (2:40 am) · 59 replies

Hi all,

I am developing with the TGE for Asia market. That is, it is required to show Chinese, Japanese or Korean.

I have already hack into the source code with a quite stupid method. Normally TGE create 256 characters bitmap for each font type, and I changed to use 128 alphabet character and with 0x8000 ~ 0xfffe double character, that is, the table has changed from 256 to 65536.

But with this method, there is lot of bitmap to pack. With only 256 characters, it is only one sheet, however, with 65536 charactes, it is around 53 to 73 sheets.

My question is, will those sheet consume 3D display card's RAM?
Moreover, due to there are so many sheets, drawing a string will have so many time of:

< in dgl.cc >
if(newObj != lastTexture)
{
if(currentPt)
{
glBindTexture(GL_TEXTURE_2D, lastTexture->texGLName);
glDrawArrays( GL_QUADS, 0, currentPt );
currentPt = 0;
}
lastTexture = newObj;
}

Will it cause too much performance drop?

Samme
#21
03/27/2005 (6:59 pm)
Lot of things to do....
In gFont.h, you have to modify CharInfo:
struct CharInfo {
      S16 bitmapIndex;    // Note: -1 indicates character is NOT to be
                          //  rendered, i.e., \n, \r, etc.
      U16  xOffset;        // x offset into bitmap sheet
      U16  yOffset;        // y offset into bitmap sheet
      U16  width;          // width of character (pixels)
      U16  height;         // height of character (pixels)
      S16  xOrigin;
      S16  yOrigin;
      S16  xIncrement;
      U8  *bitmapData;    // temp storage for bitmap data	
   };

You have to add more than 8 thousands of symbols and characters, so U16 and S16 will help you store them safely.

Also, remap table has to enlarge the size.....
S16             mRemapTable[65536];    // - Index remapping

Ok, now I have left some problems to you. Since we modify these information, lots of functions belong to gFont have to change their parameters' sizes, you will find them easily.
Don't complain, I have compiled again and again also.
#22
03/27/2005 (7:04 pm)
Puzzled!!!
could tell me the exactly position where these code added in the winfont.cc?
#23
03/27/2005 (7:09 pm)
Quote:
Puzzled!!!
could tell me the exactly position where these code added in the winfont.cc?
Hey, it's easy!
Just below
for(S32 i = 32; i < 256; i++)
	{
	   if(GetGlyphOutline(
	      fontHDC,					// handle of device context 
	      i,							// character to query 
	      GGO_GRAY8_BITMAP,		// format of data to return 
	      &metrics,				// address of structure for metrics 
	      sizeof(scratchPad),	// size of buffer for data 
	      scratchPad,				// address of buffer for data 
	      &matrix 					// address of transformation matrix structure  
	      ) != GDI_ERROR)
	   {
	      glyphCount++;
	      U32 rowStride = (metrics.gmBlackBoxX + 3) & ~3; // DWORD aligned
	  	   U32 size = rowStride * metrics.gmBlackBoxY;
	      for(U32 j = 0; j < size; j++)
	      {
	         U32 pad = U32(scratchPad[j]) << 2;
	         if(pad > 255)
	            pad = 255;
	         scratchPad[j] = pad;
	      }
	      S32 inc = metrics.gmCellIncX;
	      if(inc < 0)
	         inc = -inc;
	      retFont->insertBitmap(i, scratchPad, rowStride, metrics.gmBlackBoxX, metrics.gmBlackBoxY, metrics.gmptGlyphOrigin.x, metrics.gmptGlyphOrigin.y, metrics.gmCellIncX);
	   }
	   else
	   {
	      char b = i;
	      SIZE size;
	      GetTextExtentPoint32(fontHDC, &b, 1, &size);
	      if(size.cx)
	         retFont->insertBitmap(i, scratchPad, 0, 0, 0, 0, 0, size.cx);
	   }
	}
And above
retFont->pack(textMetric.tmHeight, textMetric.tmAscent);
#24
03/27/2005 (7:14 pm)
Don't interrupt until I finish it, please.

In function pack of gFont.cc, delete:
Point2I curSheetSize(256, 256);
add:
Point2I curSheetSize(2048, 2048);
#25
03/27/2005 (7:21 pm)
Almost done!!!
In function dglDrawTextN of dgl.cc, have you seen the "for" loop?

for (c = *str; str < endStr; c = *(++str))
......

Now we have to change it to "while" loop. Before that, we declare:
U16          c;

Ok, working on!
#26
03/27/2005 (7:24 pm)
while (str < endStr)
   {
		if(IsDBCSLeadByte(*str))
		{
			c = ((*str)<<8) | (*(str+1));
			str+=2;
		}
		else
		{
			c = *str++;
		
			// We have to do a little dance here since \t = 0x9, \n = 0xa, and \r = 0xd
			if ((c >=  1 && c <=  7) ||
				(c >= 11 && c <= 12) ||
				(c == 14)) 
			{
				// Color code
				if (colorTable) 
				{
					static U8 remap[15] = 
					{ 
						0x0,
						0x0, 
						0x1, 
						0x2, 
						0x3, 
						0x4, 
						0x5, 
						0x6, 
						0x0, 
						0x0,
						0x0, 
						0x7, 
						0x8,
						0x0,
						0x9 
					};

					U8 remapped = remap[c];
					// Ignore if the color is greater than the specified max index:
					if ( remapped <= maxColorIndex )
					{
						const ColorI &clr = colorTable[remapped];
						currentColor = sg_bitmapModulation = clr;
					}
				}
				continue;
			}
			// reset color?
			if ( c == 15 )
			{
				currentColor = sg_textAnchorColor;
				sg_bitmapModulation = sg_textAnchorColor;
				continue;
			}

			// push color:
			if ( c == 16 )
			{
				sg_stackColor = sg_bitmapModulation;
				continue;
			}

			// pop color:
			if ( c == 17 )
			{
				currentColor = sg_stackColor;
				sg_bitmapModulation = sg_stackColor;
				continue;
			}
		}

      // Tab character
      if( !font->isValidChar( c ) ) 
      {
         if ( c == '\t' ) 
         {
            const GFont::CharInfo &ci = font->getCharInfo( ' ' );
            pt.x += ci.xIncrement * GFont::TabWidthInSpaces;
         }
         continue;
      }

      const GFont::CharInfo &ci = font->getCharInfo(c);
      TextureObject *newObj = font->getTextureHandle(ci.bitmapIndex);
      if(newObj != lastTexture)
      {
         if(currentPt)
         {
            glBindTexture(GL_TEXTURE_2D, lastTexture->texGLName);
            glDrawArrays( GL_QUADS, 0, currentPt );
            currentPt = 0;
         }
         lastTexture = newObj;
      }
      if(ci.width != 0 && ci.height != 0)
      {
         pt.y = ptDraw.y + font->getBaseline() - ci.yOrigin;
         pt.x += ci.xOrigin;
         
         F32 texLeft   = F32(ci.xOffset)             / F32(lastTexture->texWidth);
         F32 texRight  = F32(ci.xOffset + ci.width)  / F32(lastTexture->texWidth);
         F32 texTop    = F32(ci.yOffset)             / F32(lastTexture->texHeight);
         F32 texBottom = F32(ci.yOffset + ci.height) / F32(lastTexture->texHeight);

         F32 screenLeft   = pt.x;
         F32 screenRight  = pt.x + ci.width;
         F32 screenTop    = pt.y;
         F32 screenBottom = pt.y + ci.height;

         vert[currentPt++].set(screenLeft, screenBottom, texLeft, texBottom, currentColor);
         vert[currentPt++].set(screenRight, screenBottom, texRight, texBottom, currentColor);
         vert[currentPt++].set(screenRight, screenTop, texRight, texTop, currentColor);
         vert[currentPt++].set(screenLeft, screenTop, texLeft, texTop, currentColor);

         pt.x += ci.xIncrement - ci.xOrigin;
      }
      else
         pt.x += ci.xIncrement;
   }
#27
03/27/2005 (7:34 pm)
Ok, we finished codes part. When you use this engine, maybe you still can't see Chinese, this is because you need to delete .gft files in folder common/ui/cache.

I am working on inputing Chinese and which is easier part. Thank you Samme Ng, you helped me a lot.

BTW, I don't think Torque is suitable for medium and large entertainment online games without change any codes, you guys have to do much homework and some technique problems is critical. They have to be solved!!!
#28
03/27/2005 (7:51 pm)
You should upload your codes include all source files and changed functions as code resource.
if it's really does work,i think it will help many many chinese developers working with TGE .
BTW:after i changed codes as you attached above,is there something more needed to do before compiling the engine?
#29
03/27/2005 (9:47 pm)
I can't recall them all in detail and just give you the most important part, some problems left will be solved easily.
My solution is not a perfect one and I still working on inputing Chinese. Before I have finished them, maybe you will give us a better answer, won't you?
#30
03/27/2005 (10:55 pm)
I must do something wrong.
VC6 can't compile the engine,when running cl.exe corrupted.
thirst for your solution about how to get TGE supported simplified Chinese!!!
#31
03/27/2005 (11:12 pm)
Use VC7.
#32
03/28/2005 (1:42 am)
Thanks Vincent! As I am busy (and lazy) to put all the codes here. :P

Since there are quite a number of changes required for fully Chinese support, I have made many changes and really can't recall them all.

For your current work, IME, be careful of VK_BACK during compositing. Normally VK_BACK is generated when you delete a compositing character, however, it will delete the GuiTextEditCtrl content also, check for VK_PROCESS?? to avoid it.
#33
03/28/2005 (4:59 pm)
VC7 can't work it out too,error C3861: "IsDBCSLeadByte occured when building dgl.cc

i appreciate you help me solve the problem ,and it will be helpful for all asia developer who haven't adequate programing skill working with TGE
#34
03/28/2005 (5:07 pm)
In dgl.cc, you should add
#include <windows.h>

Mmmmm, you will find the answer in MSDN by yourself........
#35
03/28/2005 (7:52 pm)
0 errors ,1 warning. warning LNK4075,the MSDN says ignore option1.
sorry ,i am totally innocent about VC++,i fixed the options in the tools>options>debug>edit&continue,but it doesn't work too.
when i used the torque_demo_debug.exe ,it alway arouse memory leak

please forgive my impatient question,i working on a urgent project which is need display many many chinses words.i search the GG many many times and only u give a solution about that.
#36
03/28/2005 (8:40 pm)
Hi Yanling Wu:
Me too! Run torque_demo_debug.exe after pop a error message about memory!

Hi Vincent Yang,Samme Ng:

Thanks in advance! And this is a great resource! Thanks so much!
#37
03/28/2005 (9:37 pm)
In function read and write of gFont.cc, find:
for(i = 0; i < 256; i++)
      io_rStream.read(&mRemapTable[i]);
and
for(i = 0; i < 256; i++)
      stream.write(mRemapTable[i]);

Change to
for(i = 0; i < 65536; i++)
      io_rStream.read(&mRemapTable[i]);
and
for(i = 0; i < 65536; i++)
      stream.write(mRemapTable[i]);
#38
03/28/2005 (11:20 pm)
The warning LNK4075 still occurs after building torquedemo.
i checked again the steps to make sure i do exactly as what u posted.
maybe there still have some parameters' sizes haven't changed
maybe it's hard for u to recall these steps,but i think u can zip your changed source files(winfont.cc;gFont.h,gFont.cc.etc),and upload zip file to GG,and we just download it and overwrite ,then it will be ok when building the TGEdemo again.

and thanks for your help sincerely
#39
03/29/2005 (12:39 am)
Since it is just a warning, you executable should be built. Is there any problem during execution?

Is that warning occured after you have applied the changes? Have you change the project properities also?
#40
03/29/2005 (1:00 am)
Just some work of our game:
108.opusgame.com
It is Chinese-only web site.

You can see some pictures from the game.