FPS Drop when walking on slopes

Godot 4
Hello. I’m prototyping an FPS game and i notice that my Frames Per Second drop whenever i walk on a slope that i created for testing, and the collision object on the slope directly influences that (trimesh auto-created collision is the fastest one)
I think the cause of this may be physics_process() being called every frame whenever i walk on the slope, but is there any good workaround on this?
Thanks in advance.