Camera Moves on Collision

I have an object that contains a camera and an avatar model. The camera is in a position for a third person perspective of the model. I have other objects in the scene with rigid components so the avatar can bump into these objects. When a collision occurs my camera moves from the position I have it set to some other position. Is there a way to get the camera to stay where it is on a collision?