Frame stuttering problems

Godot Version

4.6.2

Question

I keep getting issues where built up 3d scenes will stutter and oscillate between high FPS and zero FPS. The problem happens after I add an asset and then continues after I have deleted the asset. Checking the profiler shows some weird spikes that seem to be caused by various things like the audio player (i.e process taking 3000ms) but deleting the audio does not fix it. The same problem has happened to several scenes, I just dont know why reversing the modification doesnt fix the problem?