InvalidOperationException: ClipMapUniqueImageCache.DoRectUpdate

#1 10 days and no response, and this bug is driving me absolutely batty!!!

Could someone at GarageGames at least please acknowledge the bug (or tell me that I'm wrong and provide an example), because with this bug in place RAW terrain is broken. Even the very brief example in The Complete Guide to Torque X is broken and doesn't work, and nowhere have I been able to find an example that works.

I would really, really like to be able to use the great tools like L3DT Pro and FreeWorld3D that I paid good money for in order to be able to generate terrain for use in my game.

#2 Hey takuan - I will look into the issues, as I need to add some raw texture in my next title, and see if I cn help out - give me a day or so.

can you shoot reply with a snapshot of the code you are using to create raw terrain?


sean

#3 THAT WOULD BE SO AWESOME!!!!

Thanks!

btw... I'm at work at the moment, but the code is nearly identical to that found on the downloadable code found on the The Complete Guide to Torque X book's website.

Edit

P.S: I wanted to send you a PM, but we don't seem to have that ability here... I am willing to offer a code bounty if that would help :)

Second Edit
This is the code from the book website, which has an error in it (something about invalid indices if I remember right), but even when I fixed the error I had problems :

public void CreateTerrain()
{
//find the physics resolver
RigidCollisionManager rigidManager =
TorqueObjectDatabase.Instance.FindObject<RigidCollisionManager>("RigidManager");

//create the terrain
XTerrain objTerrain = new XTerrain();
objTerrain.Name = "Terrain";
objTerrain.HorizontalScale = 8;
objTerrain.VerticalScale = 1;
objTerrain.LevelZeroError = 0;
objTerrain.ViewError = 2;
objTerrain.LODLevels = 4;
objTerrain.LODError = 0.5f;
objTerrain.Repeat = true;
objTerrain.Visible = true;

//give the terrain presence in the scene
T3DSceneComponent componentScene = new T3DSceneComponent();
componentScene.Position = new Vector3(0, 0, 0);
componentScene.SceneGroup = "Terrain";
objTerrain.Components.AddComponent(componentScene);
RigidMaterial materialRigid = new RigidMaterial();
materialRigid.Restitution = 0f;
materialRigid.KineticFriction = 1f;
materialRigid.StaticFriction = 1f;

//give the terrain some physics attributes
T3DRigidComponent componentPhysics = new T3DRigidComponent();
componentPhysics.SceneGroupName = "Terrain";
componentPhysics.GravityScale = 0f;
componentPhysics.RigidManager = rigidManager;
componentPhysics.RigidMaterial = materialRigid;
componentPhysics.Immovable = true;

//componentPhysics.CollisionBody.AddCollisionShape(new CollisionXTerrainShape());
objTerrain.Components.AddComponent(componentPhysics);

// ------ SPECIFY A TERRAIN DATA TYPE ------
RAWTerrainData terrainData = new RAWTerrainData();
terrainData.TerrainFilename = "data/terrains/l3dt_generated.raw";
terrainData.LightMapFilename = "data/terrains/l3dt_generated_LM.jpg";
terrainData.UniqueTextureFilename = "data/terrains/l3dt_generated_TX.jpg";
terrainData.DetailMaterial = "TerrainDetailMaterial";
objTerrain.Data = terrainData;

//register the terrain
TorqueObjectDatabase.Instance.Register(objTerrain);
}

#4 @Takuan -
I did not have your acutal generated terrain or textures - however - I think you are just about there -
Try this:

//find the physics resolver
            RigidCollisionManager rigidManager = TorqueObjectDatabase.Instance.FindObject<RigidCollisionManager>("RigidManager");

            //SimpleMaterial
            SimpleMaterial material = new SimpleMaterial();
            material.TextureFilename = @"data\terrains\terrainTX.jpg";

            //Terrain Data
            RAWTerrainData terrainData = new RAWTerrainData();
            terrainData.TerrainFilename = @"data\terrains\terrain.raw";
            terrainData.LightMapFilename = @"data\terrains\terrainLM.jpg";
            terrainData.UniqueTextureFilename = @"data\terrains\terrainTX.jpg";

            //create the terrain
            XTerrain objTerrain = new XTerrain();
            objTerrain.Name = "Terrain";
            objTerrain.Position = new Vector3(0, 0, 0);
            objTerrain.HorizontalScale = 8;
            objTerrain.VerticalScale = 1;
            objTerrain.Repeat = true;
            objTerrain.RepeatX = true;
            objTerrain.RepeatY = true;
            objTerrain.Visible = true;
            objTerrain.Material = material;
            objTerrain.LevelZeroError = 0;
            objTerrain.ViewError = 2;
            objTerrain.LODError = 0.5f;
            objTerrain.LevelZeroError = 0;
            objTerrain.Pool = false;
            objTerrain.PoolWithComponents = false;
            objTerrain.IsTemplate = false;
            objTerrain.IsPersistent = false;
            objTerrain.Notched = false;
            objTerrain.Data = terrainData;

            //Component Scene Component
            T3DSceneComponent componentScene = new T3DSceneComponent();
            componentScene.Position = new Vector3(1024, 1036, 300);
            componentScene.SceneGroup = "Terrain";
            objTerrain.Components.AddComponent(componentScene);

            //give the Terrain
            T3DRigidComponent componentPhysics = new T3DRigidComponent();
            componentPhysics.SceneGroupName = "Terrain";
            componentPhysics.GravityScale = 1.5f;
            componentPhysics.Mass = 10.0f;
            componentPhysics.RigidManager = rigidManager;
            componentPhysics.CollisionBody.AddCollisionShape(new CollisionBoxShape());
            objTerrain.Components.AddComponent(componentPhysics);

            //Register
            TorqueObjectDatabase.Instance.Register(objTerrain);

#5 Wow, that's pretty great that it seems to render without throwing exceptions now, thanks a lot!!!

Physics doesn't seem to work, though, so I'm trying to track that down now :)


(It's not yet obvious to me what I was doing wrong, but at least now I have working code to compare to!!!)

#6 Takuan,

Hey there,
I am still working on this - I have found a bug (initally in TX) that fails when loading the UniqueTextureFilename - so the actual terrtain does not get intialized.

the issue is that the engine - in Xterrain.cs calls -
image = (Bitmap)Bitmap.FromFile(UniqueTextureFilename);
This will not work, because all images are converted to xnb's.

It should be something like this:
Resource tmpTex = ResourceManager.Instance.LoadTexture(filename);
image = (tmpTex.Instance as Texture2D).Width;

#7 Okay, so forget everything I said about problems with physics while using RAW terrain :) You apparently can't enable physics through pure code, because there are initialization steps that happen through the XML loading process that don't happen any other way, but I can work around that by using XML to define the RAW terrain once the ClipMapUniqueImageCache class is fixed.

This can be verified by changing the XTerrain.OnRegister function as follows :

// if a material wasn't set on the terrain, get one from the terrain data
	if ( _material == null )
	{
		_createdMaterial = true;
		//_material = _terrainData.CreateMaterial();
		// Generate a SimpleMaterial instead, doesn't suffer from InvalidOperationException
		_material = new SimpleMaterial();
		( (SimpleMaterial)_material ).TextureFilename = this._terrainData.UniqueTextureFilename;
	}

By forcing the terrain to use a SimpleMaterial instead, I was able to verify that all other functionality seems intact, and that the problem lies with just ClipMapUniqueImageCache, which I wasn't clear on before.

That gives me a couple of questions, though...

1) Maybe I don't really understand the purpose of this class, but my understanding is that it allows you to use a single texture for the entire terrain, and that this texture can be arbitrarily large, because the class is responsible for maintaining an 'on-demand detail' kind of internal representation. Even if this is true, it would seem that there are hard upper limits on the size of the texture which are imposed by XNA/DirectX. For instance, the XNA ContentImporter cannot handle even an 8192x8192 image, and the the compact framework used by the XBOX360 directly specifies a limit of 8K. In practice, I found that I wasn't able to load anything higher than 4096x4096, which would have insufficient detail for an entire terrain. Large textures also take excessive amounts of memory, which leads me to...

2) Is it possible for RAW terrain to use blending as an alternative? I'm trying to figure out if that's possible and how it might be implemented, but I'm once again suffering from the lack of available tutorials. It sometimes appears that the only terrain that is officially supported is TGE .ter files, and nothing else has been tested or effectively documented. If it is possible, it seems like with L3DT's ability to generate an attribute map I might be able to come up with a way to get blending information into a format that TX would understand. Hardly ideal, but it could be an interesting project, at least.

3) Where is the TGE .ter file format specification? I know I've seen it before in the past, but it's very difficult to find Torque documentation sometimes, and the TDN website appears to be completely down for the last two days. I ask because I am quite curious as to whether it's feasible for me to write custom converters and exporters to generate .ter files. Just a thought.

#8 Takuan -
I think you are correct, the upper limits on the xbox are 8kx8k - I think I have tried that before with success, I know it was certainly over 4x4k.

Great news, I am at the office still, I am a pure code guy - so I want to be able to create the terrain via code -
can you send me your xml for your xterrain creation?
Your error above is because of the following call in Xterrain.cs:
image = (Bitmap)Bitmap.FromFile(UniqueTextureFilename);

however when you fix that (fix above) you are still getting errors with physics.

Sean

#9 I too prefer pure code, so I'm super pleased to hear how persistent you are :)

<Terrain type="GarageGames.Torque.T3D.XTerrain" name="Terrain">
	<Position>
		<X>0</X>
		<Y>0</Y>
		<Z>0</Z>
	</Position>
	<HorizontalScale>8</HorizontalScale>
	<VerticalScale>1</VerticalScale>
	<Repeat>false</Repeat>
	<RepeatX>false</RepeatX>
	<RepeatY>false</RepeatY>
	<Visible>true</Visible>
	<Data type="GarageGames.Torque.T3D.RAWTerrainData">
		<TerrainFilename>data/terrains/land_HF256.raw</TerrainFilename>
		<LightMapFilename>data/terrains/land_LM.jpg</LightMapFilename>
		<UniqueTextureFilename>data/terrains/land_TX.jpg</UniqueTextureFilename>
		<TexturePathSubstitution>data/terrains</TexturePathSubstitution>
		<DetailMaterial>
		</DetailMaterial>
	</Data>
	<ViewError>2</ViewError>
	<LODError>0.5</LODError>
	<LevelZeroError>0</LevelZeroError>
	<Pool>false</Pool>
	<PoolWithComponents>false</PoolWithComponents>
	<IsTemplate>false</IsTemplate>
	<IsPersistent>false</IsPersistent>
	<Components>
		<T3DSceneComponent type="GarageGames.Torque.T3D.T3DSceneComponent" name="">
			<UseOwnerObjectType>true</UseOwnerObjectType>
			<Visible>true</Visible>
			<VisibilityLevel>1</VisibilityLevel>
			<Position>
				<X>0</X>
				<Y>0</Y>
				<Z>0</Z>
			</Position>
			<Rotation>
				<X>0</X>
				<Y>0</Y>
				<Z>0</Z>
				<W>1</W>
			</Rotation>
			<Scale>
				<X>1</X>
				<Y>1</Y>
				<Z>1</Z>
			</Scale>
		</T3DSceneComponent>
		<RigidComponent type="GarageGames.Torque.T3D.T3DRigidComponent">
			<RenderCollisionBounds>false</RenderCollisionBounds>
			<CollisionShapes>
				<CollisionShape>
					<Shape type="GarageGames.Torque.T3D.RigidCollision.CollisionXTerrainShape" />
				</CollisionShape>
			</CollisionShapes>
			<GravityScale>0.0</GravityScale>
			<Immovable>true</Immovable>
			<RigidManager nameRef="RigidManager" />
			<RigidMaterial type="GarageGames.Torque.T3D.RigidCollision.RigidMaterial">
				<Restitution>0.0</Restitution>
				<KineticFriction>0.0</KineticFriction>
				<StaticFriction>0.0</StaticFriction>
			</RigidMaterial>
		</RigidComponent>
	</Components>
</Terrain>

#10

Quote:Your error above is because of the following call in Xterrain.cs:
image = (Bitmap)Bitmap.FromFile(UniqueTextureFilename);

I guess I still am not understanding. I replaced that line with the following code :

Resource<Texture> tmpTex = ResourceManager.Instance.LoadTexture( _uniqueTextureFilename );
	imageWidth = ( tmpTex.Instance as Texture2D ).Width;

I see absolutely no difference in operation. I still get the InvalidOperationException in ClipMapUniqueImageCache.DoRectUpdate(), and am not understanding why loading the bitmap differently would have any effect in any case, as it's just being loaded in order to calculate the image width and is discarded immediately afterwards.

What am I missing here?

Edit

On a whim, I replaced the code in ClipMapUniqueImageCache.Initialize which uses System.Drawing.Bitmap with equivalent code which uses Texture2D loading code, and still see no difference in behavior. In either case, the image is being loaded only in order to fill a uint[] array with color data, so I still don't understand how the method of loading is significant.

#11 In researching the exact error, it seems that this problem is most often fixed by various incarnations of Device.Vertices[ i ].SetSource( null, 0, 0 ); (see here and here for examples), so I decided to try this myself by adding the following method to the ClipMapEffect class:

public override void CleanupEffect()
	{

		GFXDevice.Instance.Device.Indices = null;

		for( int i = 0; i < _textures.Length; i++ )
		{
			GFXDevice.Instance.Device.Vertices[ i ].SetSource( null, 0, 0 );
			GFXDevice.Instance.Device.Textures[ i ] = null;
		}

		_ClearParameters();
		base.CleanupEffect();

	}

This does completely eliminate the exception, though it doesn't help with the odd rendering artifacts I see, and which I didn't mention previously because I considered it far less important than a fatal exception.

In any case, it seems progress is being made :)

The only problem I have remaining (besides the ability to create terrain in pure code, which Sean is working on) is the rendering artifacts I see when I use RAW terrain and the ClipMapUniqueImageCache class. Note the black edge borders :

#12 I should specifically note that the rendering artifacts in that snapshot do not occur when I use SimpleMaterial instead, as I noted in a previous post.

#13 I take it back... The black artifacts aren't the only problem. Comparing that image above to the same terrain using a SimpleMaterial instead (with the same texture file), there is about a 30x difference in texture quality. SimpleMaterial is much, much, much more detailed.

I don't know what's going on :(