The problem could be that you are moving a rigid body with set pos, rigid bodies are meant to be moved with forces and managed by the physics engine.
Not sure what happens here exactly but the engine may be trying to work over the bodies and get bad collision data.
So, if not using physics, don't use bodies (at least, not rigid bodies)