I just had this issue as well, and I searched through the source code to find the cause.
The problem appears to be that when the window is in windowed mode, the window decor isn't taken into account to correct the position of the window.
OS.get_real_window_size() - OS.get_window_size() will give the total decor on each axis, so you must divide by 2 to get the amount per side.
Because the window has a title bar, the amount of decor on the top is much greater than on the bottom (the bottom only has the border and shadow), so dividing by 2 doesn't work. There you'd need to get the height of the title bar and subtract it from the y value of the calculated decor.
The problem (at least when I tested) seemed to only affect the horizontal position, so just calculating the horizontal decor and dividing by 2, then subtracting that from the origin (0,0) will give you the position for the window so that it is correctly in the top corner of the screen.
var window_decor = OS.get_real_window_size() - OS.get_window_size()
var window_position = Vector2(0,0) - window_decor.x/2
I accidentally looked through the Godot 4.0 source code first, so I believe this issue is taken care of because they have a comment that specifically mentions that the window decor is a problem in the old approach. But, I would need to either test it out in 4.0 or read more carefully to make sure.