Okay, I made it, but it surely is not perfect. From the players perspective it could be weird that a button bounces them back a bit (or bounces other rigidbody2d) + in an edge case if you collide with it at enough high velocity it will go flying to the side. Also I would really like to not unnecessarily overwork the engine.
onready var start = position
if position != start:
state.set_linear_velocity(Vector2((start.x - position.x) * 400, state.get_linear_velocity().y))