If you use
You're trying to delete a directory which isn't empty, because it contains the executable. That doesn't work. You need to delete the executable file first, and then the directory.
In addition to that, I'm not sure whether you need to pass Directory.remove a prefix that tells it to use your actual file system root, because Directory.remove might work relative to your res:// folder by default. Probably isn't a problem, though, since OS.getexecutablepath will hopefully return an absolute path that's recognized by Directory.remove as such.
Note that depending on which version of Godot you're using, the semantics of Directory.remove work out slightly differently.