Pickable cube system has lots of problems

Godot Version

4.1.3

Question

So I have a pickable cube system where you press e to pick it up and move it around. The problem is when the player picks it up they can still collide with it, therefore causing things such as flight.
Do you have any suggestions as to what I can do to make it so the world can collide with the cube but the player can’t while it is “picked?”
The code:

var picked
var pull_power = 15

# Get the gravity from the project settings to be synced with RigidBody nodes.


func on_picked():
	var collider = ray.get_collider()
	if collider != null and collider is RigidBody3D:
		picked = collider
func on_dropped():
	if picked != null:
		picked = null
#Part in the physics process function
if picked != null:
			var a = picked.global_transform.origin
			var b = hand.global_transform.origin
			picked.set_linear_velocity((b-a)*pull_power)
			picked.set_global_rotation(Vector3(-90,0,0))

Just set up the collision_mask of the cube to be on two (or more) different collision layers, one for the player and one (or more) for the “world”? When the player picks it up, update the mask by removing the bit for the player layer. When it’s dropped again, reset the bit.

That would be good, but the only problem is that if a player walks into a wall the camera would clip into the cube. I know that that isn’t the most prevalent of problems and is only really for ux but would there be a good way to scale the cube down as you get close to the wall?
(Obviously in addition to what you said before)