Hmm… you don’t seem to be showing all the related code for the snippet you’re showing.
A move_direction is being utilized but it’s definition is not being shown. I assume move_direction is, like you said, based on the camera rig’s position – but in what way?
In order to help you solve your problem, we need more explicit information; not just “it is based on the camera position”. It is often the case that mistakes are invisible to those who make them.
Leaving that aside, you may find it useful to read through a similar post concerned with orbital motion:
3D rigidbody movement around a sphere in Godot4 | Post #18
In the topic above, OP and I found a way to solve his problem of creating a character controller capable of walking on planets. While it is not your specific use-case, it should still apply.
Hopefully you can provide more information regarding your issue. Let me know if you have any questions regarding this specific problem or that of the linked topic above.