Supporting Double-Byte character
by Samme Ng · in Torque Game Engine · 12/22/2003 (2:40 am) · 59 replies
Hi all,
I am developing with the TGE for Asia market. That is, it is required to show Chinese, Japanese or Korean.
I have already hack into the source code with a quite stupid method. Normally TGE create 256 characters bitmap for each font type, and I changed to use 128 alphabet character and with 0x8000 ~ 0xfffe double character, that is, the table has changed from 256 to 65536.
But with this method, there is lot of bitmap to pack. With only 256 characters, it is only one sheet, however, with 65536 charactes, it is around 53 to 73 sheets.
My question is, will those sheet consume 3D display card's RAM?
Moreover, due to there are so many sheets, drawing a string will have so many time of:
< in dgl.cc >
if(newObj != lastTexture)
{
if(currentPt)
{
glBindTexture(GL_TEXTURE_2D, lastTexture->texGLName);
glDrawArrays( GL_QUADS, 0, currentPt );
currentPt = 0;
}
lastTexture = newObj;
}
Will it cause too much performance drop?
Samme
I am developing with the TGE for Asia market. That is, it is required to show Chinese, Japanese or Korean.
I have already hack into the source code with a quite stupid method. Normally TGE create 256 characters bitmap for each font type, and I changed to use 128 alphabet character and with 0x8000 ~ 0xfffe double character, that is, the table has changed from 256 to 65536.
But with this method, there is lot of bitmap to pack. With only 256 characters, it is only one sheet, however, with 65536 charactes, it is around 53 to 73 sheets.
My question is, will those sheet consume 3D display card's RAM?
Moreover, due to there are so many sheets, drawing a string will have so many time of:
< in dgl.cc >
if(newObj != lastTexture)
{
if(currentPt)
{
glBindTexture(GL_TEXTURE_2D, lastTexture->texGLName);
glDrawArrays( GL_QUADS, 0, currentPt );
currentPt = 0;
}
lastTexture = newObj;
}
Will it cause too much performance drop?
Samme
#22
could tell me the exactly position where these code added in the winfont.cc?
03/27/2005 (7:04 pm)
Puzzled!!! could tell me the exactly position where these code added in the winfont.cc?
#23
Just below
03/27/2005 (7:09 pm)
Quote:Hey, it's easy!
Puzzled!!!
could tell me the exactly position where these code added in the winfont.cc?
Just below
for(S32 i = 32; i < 256; i++)
{
if(GetGlyphOutline(
fontHDC, // handle of device context
i, // character to query
GGO_GRAY8_BITMAP, // format of data to return
&metrics, // address of structure for metrics
sizeof(scratchPad), // size of buffer for data
scratchPad, // address of buffer for data
&matrix // address of transformation matrix structure
) != GDI_ERROR)
{
glyphCount++;
U32 rowStride = (metrics.gmBlackBoxX + 3) & ~3; // DWORD aligned
U32 size = rowStride * metrics.gmBlackBoxY;
for(U32 j = 0; j < size; j++)
{
U32 pad = U32(scratchPad[j]) << 2;
if(pad > 255)
pad = 255;
scratchPad[j] = pad;
}
S32 inc = metrics.gmCellIncX;
if(inc < 0)
inc = -inc;
retFont->insertBitmap(i, scratchPad, rowStride, metrics.gmBlackBoxX, metrics.gmBlackBoxY, metrics.gmptGlyphOrigin.x, metrics.gmptGlyphOrigin.y, metrics.gmCellIncX);
}
else
{
char b = i;
SIZE size;
GetTextExtentPoint32(fontHDC, &b, 1, &size);
if(size.cx)
retFont->insertBitmap(i, scratchPad, 0, 0, 0, 0, 0, size.cx);
}
}And aboveretFont->pack(textMetric.tmHeight, textMetric.tmAscent);
#24
In function pack of gFont.cc, delete:
03/27/2005 (7:14 pm)
Don't interrupt until I finish it, please.In function pack of gFont.cc, delete:
Point2I curSheetSize(256, 256);add:
Point2I curSheetSize(2048, 2048);
#25
In function dglDrawTextN of dgl.cc, have you seen the "for" loop?
Now we have to change it to "while" loop. Before that, we declare:
Ok, working on!
03/27/2005 (7:21 pm)
Almost done!!!In function dglDrawTextN of dgl.cc, have you seen the "for" loop?
for (c = *str; str < endStr; c = *(++str)) ......
Now we have to change it to "while" loop. Before that, we declare:
U16 c;
Ok, working on!
#26
03/27/2005 (7:24 pm)
while (str < endStr)
{
if(IsDBCSLeadByte(*str))
{
c = ((*str)<<8) | (*(str+1));
str+=2;
}
else
{
c = *str++;
// We have to do a little dance here since \t = 0x9, \n = 0xa, and \r = 0xd
if ((c >= 1 && c <= 7) ||
(c >= 11 && c <= 12) ||
(c == 14))
{
// Color code
if (colorTable)
{
static U8 remap[15] =
{
0x0,
0x0,
0x1,
0x2,
0x3,
0x4,
0x5,
0x6,
0x0,
0x0,
0x0,
0x7,
0x8,
0x0,
0x9
};
U8 remapped = remap[c];
// Ignore if the color is greater than the specified max index:
if ( remapped <= maxColorIndex )
{
const ColorI &clr = colorTable[remapped];
currentColor = sg_bitmapModulation = clr;
}
}
continue;
}
// reset color?
if ( c == 15 )
{
currentColor = sg_textAnchorColor;
sg_bitmapModulation = sg_textAnchorColor;
continue;
}
// push color:
if ( c == 16 )
{
sg_stackColor = sg_bitmapModulation;
continue;
}
// pop color:
if ( c == 17 )
{
currentColor = sg_stackColor;
sg_bitmapModulation = sg_stackColor;
continue;
}
}
// Tab character
if( !font->isValidChar( c ) )
{
if ( c == '\t' )
{
const GFont::CharInfo &ci = font->getCharInfo( ' ' );
pt.x += ci.xIncrement * GFont::TabWidthInSpaces;
}
continue;
}
const GFont::CharInfo &ci = font->getCharInfo(c);
TextureObject *newObj = font->getTextureHandle(ci.bitmapIndex);
if(newObj != lastTexture)
{
if(currentPt)
{
glBindTexture(GL_TEXTURE_2D, lastTexture->texGLName);
glDrawArrays( GL_QUADS, 0, currentPt );
currentPt = 0;
}
lastTexture = newObj;
}
if(ci.width != 0 && ci.height != 0)
{
pt.y = ptDraw.y + font->getBaseline() - ci.yOrigin;
pt.x += ci.xOrigin;
F32 texLeft = F32(ci.xOffset) / F32(lastTexture->texWidth);
F32 texRight = F32(ci.xOffset + ci.width) / F32(lastTexture->texWidth);
F32 texTop = F32(ci.yOffset) / F32(lastTexture->texHeight);
F32 texBottom = F32(ci.yOffset + ci.height) / F32(lastTexture->texHeight);
F32 screenLeft = pt.x;
F32 screenRight = pt.x + ci.width;
F32 screenTop = pt.y;
F32 screenBottom = pt.y + ci.height;
vert[currentPt++].set(screenLeft, screenBottom, texLeft, texBottom, currentColor);
vert[currentPt++].set(screenRight, screenBottom, texRight, texBottom, currentColor);
vert[currentPt++].set(screenRight, screenTop, texRight, texTop, currentColor);
vert[currentPt++].set(screenLeft, screenTop, texLeft, texTop, currentColor);
pt.x += ci.xIncrement - ci.xOrigin;
}
else
pt.x += ci.xIncrement;
}
#27
I am working on inputing Chinese and which is easier part. Thank you Samme Ng, you helped me a lot.
BTW, I don't think Torque is suitable for medium and large entertainment online games without change any codes, you guys have to do much homework and some technique problems is critical. They have to be solved!!!
03/27/2005 (7:34 pm)
Ok, we finished codes part. When you use this engine, maybe you still can't see Chinese, this is because you need to delete .gft files in folder common/ui/cache.I am working on inputing Chinese and which is easier part. Thank you Samme Ng, you helped me a lot.
BTW, I don't think Torque is suitable for medium and large entertainment online games without change any codes, you guys have to do much homework and some technique problems is critical. They have to be solved!!!
#28
if it's really does work,i think it will help many many chinese developers working with TGE .
BTW:after i changed codes as you attached above,is there something more needed to do before compiling the engine?
03/27/2005 (7:51 pm)
You should upload your codes include all source files and changed functions as code resource.if it's really does work,i think it will help many many chinese developers working with TGE .
BTW:after i changed codes as you attached above,is there something more needed to do before compiling the engine?
#29
My solution is not a perfect one and I still working on inputing Chinese. Before I have finished them, maybe you will give us a better answer, won't you?
03/27/2005 (9:47 pm)
I can't recall them all in detail and just give you the most important part, some problems left will be solved easily.My solution is not a perfect one and I still working on inputing Chinese. Before I have finished them, maybe you will give us a better answer, won't you?
#30
VC6 can't compile the engine,when running cl.exe corrupted.
thirst for your solution about how to get TGE supported simplified Chinese!!!
03/27/2005 (10:55 pm)
I must do something wrong.VC6 can't compile the engine,when running cl.exe corrupted.
thirst for your solution about how to get TGE supported simplified Chinese!!!
#31
03/27/2005 (11:12 pm)
Use VC7.
#32
Since there are quite a number of changes required for fully Chinese support, I have made many changes and really can't recall them all.
For your current work, IME, be careful of VK_BACK during compositing. Normally VK_BACK is generated when you delete a compositing character, however, it will delete the GuiTextEditCtrl content also, check for VK_PROCESS?? to avoid it.
03/28/2005 (1:42 am)
Thanks Vincent! As I am busy (and lazy) to put all the codes here. :PSince there are quite a number of changes required for fully Chinese support, I have made many changes and really can't recall them all.
For your current work, IME, be careful of VK_BACK during compositing. Normally VK_BACK is generated when you delete a compositing character, however, it will delete the GuiTextEditCtrl content also, check for VK_PROCESS?? to avoid it.
#33
i appreciate you help me solve the problem ,and it will be helpful for all asia developer who haven't adequate programing skill working with TGE
03/28/2005 (4:59 pm)
VC7 can't work it out too,error C3861: "IsDBCSLeadByte occured when building dgl.cci appreciate you help me solve the problem ,and it will be helpful for all asia developer who haven't adequate programing skill working with TGE
#34
Mmmmm, you will find the answer in MSDN by yourself........
03/28/2005 (5:07 pm)
In dgl.cc, you should add#include <windows.h>
Mmmmm, you will find the answer in MSDN by yourself........
#35
sorry ,i am totally innocent about VC++,i fixed the options in the tools>options>debug>edit&continue,but it doesn't work too.
when i used the torque_demo_debug.exe ,it alway arouse memory leak
please forgive my impatient question,i working on a urgent project which is need display many many chinses words.i search the GG many many times and only u give a solution about that.
03/28/2005 (7:52 pm)
0 errors ,1 warning. warning LNK4075,the MSDN says ignore option1.sorry ,i am totally innocent about VC++,i fixed the options in the tools>options>debug>edit&continue,but it doesn't work too.
when i used the torque_demo_debug.exe ,it alway arouse memory leak
please forgive my impatient question,i working on a urgent project which is need display many many chinses words.i search the GG many many times and only u give a solution about that.
#36
Me too! Run torque_demo_debug.exe after pop a error message about memory!
Hi Vincent Yang,Samme Ng:
Thanks in advance! And this is a great resource! Thanks so much!
03/28/2005 (8:40 pm)
Hi Yanling Wu:Me too! Run torque_demo_debug.exe after pop a error message about memory!
Hi Vincent Yang,Samme Ng:
Thanks in advance! And this is a great resource! Thanks so much!
#37
Change to
03/28/2005 (9:37 pm)
In function read and write of gFont.cc, find:for(i = 0; i < 256; i++)
io_rStream.read(&mRemapTable[i]);andfor(i = 0; i < 256; i++)
stream.write(mRemapTable[i]);Change to
for(i = 0; i < 65536; i++)
io_rStream.read(&mRemapTable[i]);andfor(i = 0; i < 65536; i++)
stream.write(mRemapTable[i]);
#38
i checked again the steps to make sure i do exactly as what u posted.
maybe there still have some parameters' sizes haven't changed
maybe it's hard for u to recall these steps,but i think u can zip your changed source files(winfont.cc;gFont.h,gFont.cc.etc),and upload zip file to GG,and we just download it and overwrite ,then it will be ok when building the TGEdemo again.
and thanks for your help sincerely
03/28/2005 (11:20 pm)
The warning LNK4075 still occurs after building torquedemo.i checked again the steps to make sure i do exactly as what u posted.
maybe there still have some parameters' sizes haven't changed
maybe it's hard for u to recall these steps,but i think u can zip your changed source files(winfont.cc;gFont.h,gFont.cc.etc),and upload zip file to GG,and we just download it and overwrite ,then it will be ok when building the TGEdemo again.
and thanks for your help sincerely
#39
Is that warning occured after you have applied the changes? Have you change the project properities also?
03/29/2005 (12:39 am)
Since it is just a warning, you executable should be built. Is there any problem during execution?Is that warning occured after you have applied the changes? Have you change the project properities also?
#40
108.opusgame.com
It is Chinese-only web site.
You can see some pictures from the game.
03/29/2005 (1:00 am)
Just some work of our game:108.opusgame.com
It is Chinese-only web site.
You can see some pictures from the game.
Torque Owner Harry Chow
In gFont.h, you have to modify CharInfo:
struct CharInfo { S16 bitmapIndex; // Note: -1 indicates character is NOT to be // rendered, i.e., \n, \r, etc. U16 xOffset; // x offset into bitmap sheet U16 yOffset; // y offset into bitmap sheet U16 width; // width of character (pixels) U16 height; // height of character (pixels) S16 xOrigin; S16 yOrigin; S16 xIncrement; U8 *bitmapData; // temp storage for bitmap data };You have to add more than 8 thousands of symbols and characters, so U16 and S16 will help you store them safely.
Also, remap table has to enlarge the size.....
Ok, now I have left some problems to you. Since we modify these information, lots of functions belong to gFont have to change their parameters' sizes, you will find them easily.
Don't complain, I have compiled again and again also.