MS4 and L3DT blended troubles
by Erik Madison · in Torque Game Engine Advanced · 09/11/2006 (12:11 pm) · 41 replies
Just prior to MS4s release, I had an Atlas2 version of stronghold up and running. I had done this via unique terrain. After MS4, it no longer worked (even with the new lightmap settings in the .mis). Now I realize how much better a blended terrain looks, so I'm trying to recreate it that way. So far, neither method seems to work anymore. I hex looked over arcticBlended.atlas, and saw the l3dt texture directorie listed. I created a new, better placed texture dir, made a new climate with those textures, and ran the below commands. Hex checking the final output, I don't even see my textures listed, and needless to say it won't load.
From the now ancient TDN article, I find some oddities.
First off, generate from tiles requires a 4th arg, which I'm assuming is 1/0 png/jpg. That at least output decent looking console messages.
This confuses me though:
So... my question is thus: WTF am I doing wrong? :) Perhaps the climate file and exact scripts used to make arcticBlended.atlas would shed some light on this, as that Atlas2 obviously loads and looks fine.
// L3DT version
atlasOldGenerateChunkFileFromRaw16("client/data/terrains/l3dt/stronghold.RAW", 512, 4, 1.0/256.0, "client/data/terrains/geometry.chu", 2.0, 3);
importOldAtlasCHU("client/data/terrains/geometry.chu", "client/data/terrains/geometry.atlas");
atlasGenerateTextureTOCFromTiles(4, "client/data/terrains/l3dt/stronghold_opacity_Alpha1/Alpha1_x%dy%d.png", "client/data/terrains/opacity.atlas", 1);
atlasGenerateTextureTOCFromTiles(4, "client/data/terrains/l3dt/LM/stronghold_LM_x%dy%d.jpg", "client/data/terrains/lightmap_png.atlas", 0);
atlasConvertTextureTOC("client/data/terrains/lightmap_png.atlas", "client/data/terrains/lightmap.atlas", 0);
atlasGenerateBlenderTerrain("client/data/terrains/stronghold.atlas", "client/data/terrains/geometry.atlas", "client/data/terrains/opacity.atlas", "client/data/terrains/lightmap.atlas");From the now ancient TDN article, I find some oddities.
First off, generate from tiles requires a 4th arg, which I'm assuming is 1/0 png/jpg. That at least output decent looking console messages.
This confuses me though:
Quote:The texturemap process outputs a unique texture, tiled or not. Whats in the LM dir is the actual lightmap. The opacity map with alphas et al is done via generate alphas. The example script (which my above script very closely follows) seems to do it the logical way, while the warning quote seems to completely contradict this.
Note: The lightmap is the group of mosiac textures generated in the "Texture Map" process and the opacity map is the image generated in the "Light Map" process. Be careful to not get these confused.
So... my question is thus: WTF am I doing wrong? :) Perhaps the climate file and exact scripts used to make arcticBlended.atlas would shed some light on this, as that Atlas2 obviously loads and looks fine.
About the author
#22
09/18/2006 (2:08 pm)
Hi Erik, thanks for that. However it will only make 4 maps. The box with "number of images" is greyed out and I can't select it. I only have the unregistered version, so maybe that might have something to do with it. I'll register it tomorrow and see if that changes anything. Thanks again.
#23
09/18/2006 (4:37 pm)
You want to make sure you're exporting PNGs. I believe you do have to register for this.
#24
Sort of off-topic but amusingly, I discovered by accident that if you use an alpha map from one of the low coverage textures that you get a black landscape with lots of random glowing spots on it. With a suitable texture under the alpha map it looks just like a charred black wasteland with lava spots popping through :-)
Dave.
09/19/2006 (12:37 am)
Hi Ben, yes that's true. I manually converted the BMPs to PNGs just to test the results. That's actually what I was thinking of when I said I'd register L3DT. Perhaps the registered version outputs one PNG instead of 4 BMPs.Sort of off-topic but amusingly, I discovered by accident that if you use an alpha map from one of the low coverage textures that you get a black landscape with lots of random glowing spots on it. With a suitable texture under the alpha map it looks just like a charred black wasteland with lava spots popping through :-)
Dave.
#25
Thanks very much for everybody's help. Greatly appreciated.
Dave.
09/19/2006 (8:11 am)
Well, got my registration code and ran through the process again with the pro version. It generated a single Alpha map. I used that and everything is fine now. The right tools for the job etc! Thanks very much for everybody's help. Greatly appreciated.
Dave.
#26
09/19/2006 (10:32 pm)
Cool - very glad you got it working! :)
#27
09/20/2006 (2:49 am)
Thanks Ben. I think I have to start another thread with another stupid question now, but I'll hack away at it for a few more hours before doing so :-)
#28
(PS absolutely stunningly superb blender terrains now I have it working. Thanks a million for the hard work).
09/20/2006 (4:52 am)
Back on the subject (so soon), when I create a high-res lightmap the terrain generation craps out with a message saying "Cannot load PNG files greater than 1024 in height". Is there a reason for this limitation? Thx.(PS absolutely stunningly superb blender terrains now I have it working. Thanks a million for the hard work).
#29
Just go to gfx/bitmapPNG.cc and change the define at the top of it to a higher number.
If you have a very large image (> 4k or so) you probably want to use JPEG as there's an efficient streaming reader implement for it (the largeJPEG console function).
Or you can export a mosaic from L3DT - it supports turning an existing image into a mosaic. That way you can continue to use PNG.
09/20/2006 (9:56 am)
Yes - we had to choose an arbitrary limit for the PNG loader and 1024 was a default (but an old one - are you sure your bitmapPNG.cc file is up to date?).Just go to gfx/bitmapPNG.cc and change the define at the top of it to a higher number.
If you have a very large image (> 4k or so) you probably want to use JPEG as there's an efficient streaming reader implement for it (the largeJPEG console function).
Or you can export a mosaic from L3DT - it supports turning an existing image into a mosaic. That way you can continue to use PNG.
#30
Yes, it's a download of HEAD from a couple of days back. I think I'll try the mosaic route to keep things lossless. I suppose it's a bit pointless making a high def map and then using JPG to store it.
Thanks for the help, good man yerself,
Dave.
09/20/2006 (12:49 pm)
Hi Ben,Yes, it's a download of HEAD from a couple of days back. I think I'll try the mosaic route to keep things lossless. I suppose it's a bit pointless making a high def map and then using JPG to store it.
Thanks for the help, good man yerself,
Dave.
#31
Thx,
Dave.
09/22/2006 (3:47 am)
Noticed there were some updates for Atlas and lighting in CVS today. I downloaded them, cleaned my project and rebuilt. The mesh and lighting now match up without flipping the mesh in L3DT. So whether it was the updates or the clean build I have no idea, but you don't need to go through all that export/import routine in L3DT anymore.Thx,
Dave.
#32
Atlas fixes:
- Added destructor to atlasClipMapBatcher.
- Fixed UV import from CHU.
- Simplified clipmap rendernote binning logic.
- Sped up unique cache texture uploads and corrected color swap.
- Made batcher a member of AtlasInstance2, instead of a temporary.
- Now imports properly from L3DT again.
- Fix for incorrect bounding box generation in some cases.
- Fix for flat terrains
09/22/2006 (10:58 am)
Probably the updates.Atlas fixes:
- Added destructor to atlasClipMapBatcher.
- Fixed UV import from CHU.
- Simplified clipmap rendernote binning logic.
- Sped up unique cache texture uploads and corrected color swap.
- Made batcher a member of AtlasInstance2, instead of a temporary.
- Now imports properly from L3DT again.
- Fix for incorrect bounding box generation in some cases.
- Fix for flat terrains
#33
09/22/2006 (7:38 pm)
Yay my terrain texture isn't blue anymore :P
#34
I've moified the defines at the top of the bitmapPng.cpp file as follows
static const U32 csgMaxRowPointers = 4096;
static png_bytep sRowPointers[4096];
I am now getting the following error creating the alpha map from the JPG using the largeJPG function.
Fatal-ISV: (\gamedev\TseM4\tse\engine\atlas\resource\atlasTexChunk.cpp @ 97) AtlasTexChunk::write - failed writing bitmap!
I also tried it without changing sRowPointers as you only mentioned the one define, but it makes no difference. The error comes from an assertion on line 97 of AtlasTexChunk.cpp which relies on the statement on line 83 I think
writeSuccess = bitmap->writePNG(*s, false);
I have traced the definition of the header for writePng back to gbitmap.h. But from there I am stumped as I am not a C programmer and was winging it this far. I thought the method itself would be in gbitmap.cpp but it's not.
Any ideas how to fix this would be appreciated,
Thanks,
Dave.
09/25/2006 (10:57 am)
Hi Ben,I've moified the defines at the top of the bitmapPng.cpp file as follows
static const U32 csgMaxRowPointers = 4096;
static png_bytep sRowPointers[4096];
I am now getting the following error creating the alpha map from the JPG using the largeJPG function.
Fatal-ISV: (\gamedev\TseM4\tse\engine\atlas\resource\atlasTexChunk.cpp @ 97) AtlasTexChunk::write - failed writing bitmap!
I also tried it without changing sRowPointers as you only mentioned the one define, but it makes no difference. The error comes from an assertion on line 97 of AtlasTexChunk.cpp which relies on the statement on line 83 I think
writeSuccess = bitmap->writePNG(*s, false);
I have traced the definition of the header for writePng back to gbitmap.h. But from there I am stumped as I am not a C programmer and was winging it this far. I thought the method itself would be in gbitmap.cpp but it's not.
Any ideas how to fix this would be appreciated,
Thanks,
Dave.
#35
If you have mondo chunks you WILL experience degraded paging performances! It's always going to be faster to bring in a few hundred k chunks rather than an 8mb monstrosity!
09/25/2006 (1:52 pm)
Sure - make the chunk buffer size bigger. It's located in AtlasChunk.h IIRC - it limits the maximum size chunk you can write out. (Since we don't know a priori what size of a chunk we'll be writing I had to make a guess - and it might be too low for some people.)If you have mondo chunks you WILL experience degraded paging performances! It's always going to be faster to bring in a few hundred k chunks rather than an 8mb monstrosity!
#36
Thanks, will give that a whirl. I'm limiting it to 4096 for now, so hopefully that will fall into the mid-range and not do too much damage. Will update and let you know later.
Hmmm, well I upped those to 4096 and it still crashes with the same error. However, I can now load 2048 alpha maps, so it *has* made a difference. So I don't understand why maps of 4096 size won't load. I checked in PSP and the image is definitely 4096 in size. I did a clean build as well to make sure, but no joy.
I upped the defines further to 8192 just for the craic to see if it would load, but I got a memory error.
Any other ideas?
Thanks again,
Dave.
09/26/2006 (2:46 am)
Hi Ben,Thanks, will give that a whirl. I'm limiting it to 4096 for now, so hopefully that will fall into the mid-range and not do too much damage. Will update and let you know later.
Hmmm, well I upped those to 4096 and it still crashes with the same error. However, I can now load 2048 alpha maps, so it *has* made a difference. So I don't understand why maps of 4096 size won't load. I checked in PSP and the image is definitely 4096 in size. I did a clean build as well to make sure, but no joy.
I upped the defines further to 8192 just for the craic to see if it would load, but I got a memory error.
Any other ideas?
Thanks again,
Dave.
#37
09/28/2006 (7:52 pm)
Did you up the chunk buffer size like I suggsted?
#38
MaxChunkSize = 8 * 4096 * 4096
I assume if I hadn't the 2048 file would have caused a prob? It went through just fine.
Thanks,
Dave.
09/29/2006 (11:15 am)
Yup, certainly did. I changed the enum in Atlaschunk.h to beMaxChunkSize = 8 * 4096 * 4096
I assume if I hadn't the 2048 file would have caused a prob? It went through just fine.
Thanks,
Dave.
#39
10/02/2006 (8:11 pm)
Can you get a callstack for me?
#40
OK, I figured out how to get the debugger working in VC++ and even how to get it to find main.cs. Ran the debugger and saw a call stack window which remained empty during debugging for some reason (and vanished at the end of debugging). Here is the output from the main window after debugging.
I had a look in platformMemory.cpp and found two defines at the top which might be affecting things
TreeNodeAllocCount = 2048
static U32 MinPageSize = 8 * 1024 * 1024
The number in TreeNodeAllocCount looks suspicious to me, but what do I know?
I wouldn't imagine a min size for anything would cause problems. I might change just the treenode count and see what happens. If you don't hear from me again you know I've fried my PC.
Well, I still have a PC but the change made no difference
Cheers,
Dave.
10/03/2006 (3:50 am)
Hi Ben, you may have read my other post about how familiar I am with C, so I guess this is a punishment exercise :-)OK, I figured out how to get the debugger working in VC++ and even how to get it to find main.cs. Ran the debugger and saw a call stack window which remained empty during debugging for some reason (and vanished at the end of debugging). Here is the output from the main window after debugging.
AtlasOldActivationHeightfield::generateNodeData - Max exceeded! May have paging issues! PlatformMemory: allocating new page, total bytes allocated so far: 20781092 (total bytes in all pages=159383620) The thread 'Win32 Thread' (0x2d1c) has exited with code 0 (0x0). The thread 'Win32 Thread' (0x2e68) has exited with code 0 (0x0). The thread 'Win32 Thread' (0x2b94) has exited with code 0 (0x0). PlatformMemory: allocating new page, total bytes allocated so far: 90068172 (total bytes in all pages=293601416) Fatal-ISV: (c:\gamedev\tsem4\tse\engine\atlas\resource\atlastexchunk.cpp @ 97) AtlasTexChunk::write - failed writing bitmap! The thread 'Win32 Thread' (0x2380) has exited with code 0 (0x0). The thread 'Win32 Thread' (0x2d9c) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2ab4) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2354) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2c48) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2c60) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x1bd4) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2ce0) has exited with code 1 (0x1). The thread 'Win32 Thread' (0x2aa4) has exited with code 1 (0x1). Fatal: (c:\gamedev\tsem4\tse\engine\interior\interiorlmmanager.cpp @ 171) InteriorLMManager::removeInterior: cannot remove base interior Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. Fatal: (c:\gamedev\tsem4\tse\engine\platform\platformmemory.cpp @ 1196) Bad block flags. The program '[11404] TSE_DEBUG.exe: Native' has exited with code 1 (0x1).
I had a look in platformMemory.cpp and found two defines at the top which might be affecting things
TreeNodeAllocCount = 2048
static U32 MinPageSize = 8 * 1024 * 1024
The number in TreeNodeAllocCount looks suspicious to me, but what do I know?
I wouldn't imagine a min size for anything would cause problems. I might change just the treenode count and see what happens. If you don't hear from me again you know I've fried my PC.
Cheers,
Dave.
Torque Owner Erik Madison
atlasGenerateTextureTOCFromTiles(1, "client/data/terrains/l3dt/MyGame_Alpha1.png", "client/data/terrains/opacity.atlas", 1);
will do the trick.
If you still have problems, check the nearly complete and updated documentation.