From the docs:
"There is no dedicated server build for Windows yet. As an alternative, you can use the
--no-window command-line argument to prevent Godot from spawning a window. Note that even with the
--no-window command-line argument, you'll need to have OpenGL support available on the Windows machine." cite
MacOS and Linux headless builds are available, however.