I had found out that my icons were showing up but only if they were large or extra-large. Otherwise, the icon displays the Godot icon.
Windows executables can bundle many icon sizes. If you only replace some sizes, the Godot icon will be kept for the other sizes.
Also, note that Windows (and other OSes) keep an icon cache, which can cause outdated icons to be displayed if you re-export an executable with the same name. To avoid this, always export your executable with a different name when testing icon changes.