Just found this. Well, it seems the only solution then is to create a subnode in the RigidBody2D and store everything in it, and scale this node instead.
I don’t think this works anymore in Godot 3.x because a RigidBody2D needs to have a direct children of type CollisionShape2D.
I also don’t think this is allowed in Godot 3.x because the documentation says that applying a scale to the CollisionShape2D child is illegal and leads to unexpected behavior. And scaling only the visible children, but not the shape obviously also doesn’t lead to the desired behavior.