I've personally found that I don't like using richtextlabels for this reason specifically. I haven't looked into the "benefits" of a tichtextlabel over a normal label node, but all I know is that label nodes can have dynamic fonts, which could easily fix your problem.
This is a pretty great demonstration of how to set up a label node with dynamic fonts.
https://www.youtube.com/watch?v=8JgJuF7imNg
What I would recommend is making the label node separate from your main project first to see if it fixes your blur issue.