Okay, I figured it out...
To remove the unwanted blue "style box" border around focused buttons, do the following:
- In the inspector for the button node, scroll down until you find "Custom Styles", expand.
- Under the focus property, set the null value to StyleBoxEmpty.
I feel like this was harder to find than it should have been, the documentation simply says "Empty stylebox (does not display anything).".