Interior Lightmaps using too much ram

#1 How much interior data do you have? Vertex lighting at least in theory should give you an overhead of something like 12 bytes/vert, which is pretty light for lightmap-esque functionality.

#2 "Vertex lighting at least in theory should give you an overhead of something like 12 bytes/vert, which is pretty light for lightmap-esque functionality."

The mission takes 150 megs of ram with vertex lighting. and 650 megs with lightmaps. Yes, vertex lighting is cheap. It's also very weak and prone to lighting errors...

"How much interior data do you have?"

Not enough to justify 500 megs of mission lighting data. Which is mostly solid shaded lightmaps anyway...

-Josh

#3 Josh how do you measure this ?
Could be interesting to know.
I have some heavy maps to .

#4 Josh,

I'm trying to understand the circumstances around your problem. I expressed a baseline for what I felt a "reasonable" overhead was, and then inquired as to whether that was inline with your observations. If I am to address this issue, I have to know what sort and how much interior data you have in your mission. I know you're working with MMO-scale content, so it's entirely likely you're pushing Torque further in this regard than anyone has before.

Did I come across as accusatory? If so, I apologize; I tend to be terse on the forums, because of the large volume of posts I produce. What I said was borne of a desire to understand and help.

Ben

#5 @Ben:

No worries...

The XP task manager reports 150 megs of memory usage when using vertex lighting and 650 megs when lightmaps are enabled.

The scene is a city with an enclosing wall... it's less complex than Tim Aste's demo city for his content pack video... I believe the scenelighting is naively creating many, many more lightmaps than need be... as in, piles of singled shaded lightmaps that could/should be shared...

It would seem mission lighting is a huge ram waster for scenes with any amount of interiors...

-Josh

#6 How many brushes does it have? How many surfaces? One big instance or many small ones? Can you post the interior or another interior that exhibits similar behavior?

I checked the code, and it appears that each interior gets a vector of vertex colors, one for alarm, one for normal. One per vert, maybe one per index (haven't read the code that closely)... So it seems like for a 64k vert interior, you're talking about a few megs of memory.

Have you tried dumping vector sizes from the interior load code?

#7 Ben,

It's the lightmaps man... you can check out the usage by loading up the Torque demo and instancing the great hall 20 or so times... everything is great until you apply the mission lighting...

Instancing dif's is awesome, it doesn't look like the lightmaps are being instanced (where possible), it's also seems like there's an awful lot of base lightmap copying going on...

-Josh

#8 From what I recall the Lightmap Manager does try to share as many lightmaps as possible between instances of the interiors. The problem is that it can only share "interior", zoned lightmaps where the ambient light isn't being let through. The "exterior" lightmaps can change from instance to instance based on its position/rotation/scale in the world so it can only be sure that the "interior" lightmaps match. If you wanted you *could* do a per pixel comparison of each lightmap every time the scene is lit and aggregate those together more intelligently but I suspect that will lead quickly to unacceptable relighting times. Once they are lit though, the lightmap information will be cached and until you change the mission the loadtimes and memory footprints will be much faster.

Couple of routes I would explore here:
1) While you are working on the mission have the interiors render in straight gl lighting (ie no lightmaps at all). Then once you have the mission solid, relight and just let it run through the per-pixel comparisons and consolidations of the lightmaps.

2) Operate with torque normally until the mission is "done" and then do a special consolidation relight.

3) Operate with torque normally and then have an external app consolidate the cached lighting file from time to time (could batch it up for every night).

4) Use lower precision and smaller lightmaps overall.

#9 Yeah, my first thought was per pixel comparison allowing a small amount of error...

It's a significant issue for interiors and definitely needs to be fixed before we ship...

Too many plates spinning at the moment...

-Josh

#10 I have such problem in my game also.

#11 TGE 1.5 should significantly reduce light map memory usage over TGE 1.3/1.4.

It's hard to tell how memory heavy your light maps are and why without seeing a few scene shots of the overall scene, the interior lighting detail, and some stats on home much light map memory it looks like torque is using.


Here are a few general rules to reduce light map memory usage:

-use portals and zones - the sun clones all outside surface light maps, without portals/zones all interior surfaces are considered outside surfaces (making copies of every light map)

-use the same interiors several times instead of creating many unique interiors - light maps are shared between instances of the same interior (but again zones/portals are needed)

-avoid using too many mission static point/spot lights - these cause cloning of interior light maps (including inside surface light maps)

-when all else fails reduce interior lighting density - increase (which lowers the quality) the lighting scale for all interiors

#12

Quote:
-avoid using too many mission static point/spot lights - these cause cloning of interior light maps (including inside surface light maps)

Woah. Isn't static point mission lights, the same as the ones placed in your CSG editor?
We've been using *only* static lights placed in the editor, and not in our CSG app. I guess that means we should change our habits, then?

#13 Hi Stefan,

Map and mission lights both bake into interior light maps, however how they do it, or more specifically when, is slightly different.

Map lights are placed in the interior using a CSG editor, and are baked into the interior light maps when the interior dif file is compiled using map2dif(+) or Constructor's export. This lighting is permanently baked into the interior and cannot be changed without editing the map file and re-compiling the dif. Because the lighting is permanently baked into the interior, all in-game instances of the interior have the exact same lighting, which allows light maps to be shared.

Mission lights are placed in-game and are baked into the light maps during the lighting phase, which lights each interior instance separately. This allows each interior instance to have unique mission lighting, even those sharing the same dif file (though map lighting will still be the same between them). This works by cloning any light map that's changed during the lighting phase, creating a unique copy for that interior instance.

For best results any lighting that's the same between all instances of the same dif should be created using map lights and permanently baked into the dif file, and any unique lighting created with mission lights - this gives you the best of both worlds, unique lighting and lowest possible light map memory usage (and also reduces relight times).

#14 Another quick note: the sun is also a mission light, which, like all mission lights, clones the light maps it interacts with. It's important to use zones and portals to prevent the sun from illuminating the entire interior and cloning all of its light maps.