I don’t quite understand what you mean by adjusting the font size to ± 4px. However, I haven’t had any problems working with very low res UIs. Have you turned antialiasing off where that’s possible? And are you using a proper pixel font?
I don’t think the editor view can be made to match the final rendering exactly but I have found it to not hinder my work.
You render your UI at a higher resolution with a normal font size. Then your pixel art is rendered by the SubViewport at the Gameboy Advance resolution.
Make sure your textures are imported as lossless so you don’t see compression artifacts.
You probably don’t want mipmaps in a pixel art game.
Edit: You also should use a font designed for low resolution. Sometimes referred to as pixel fonts. That alone will render fonts more crisp and may obviate SubViewport. Make sure to turn off any smoothing or anti-aliasing.