You may be using a function that takes relative position instead of global / location position.
Without the actual code you're using, I'll try to answer the best I can.
Subtract the position you've recieved from input from the position of the object you're trying to move. That will give you a relative position using the object as the origin instead of the global origin.
EDIT: After a lot more googling, try using get_global_mouse_position()
for mouse click position.