Apparently, this is indeed issue with Nvidia drivers, I saw two reported issue on Github Godot’s, and one of them recommend to downgrade Nvidia drivers to 572.70: Inconsistent Rendering Issue in Editor and Game Camera · Issue #109704 · godotengine/godot · GitHub
This solves the problem with spikes and jitter.
TIL Don’t use the latest drivers 