I'm fine with bypassing gatekeeper for now, these are just test builds.
The problem is that on an ARM mac, the normal gatekeeper dialog does not appear. Instead, another dialog appears that states the application can not be opened.
On an intel mac, it works as you described.