Hello im trying to make a custom cursor for my game how should i aproach this

In your code, the part for checking the group is fine exept for the CollisionShape3D() at the beginning. (This should actually throw an error because CollisionShape3D does not have a constructor)
You should call is_in_group on the object that the script is on, so just if is_in_group("Enemy"): (or I suppose if self.is_in_group("Enemy"): if you prefer that way) would work.

The first step doesn’t seem to be implemented though, and you still have the _on_mouse_entered(): ... from before. Or is it just in another script and you didn’t delete that part yet? In which case I wouldn’t know why the script extends Area2D which is a 2D node that doesn’t belong in a 3D world.

i use a 2d because its part of the ui and that works on 2d.

  1. DisplayServer doesn’t have the function get_mouse_cursor(). Also, as I said:
  1. Didn’t you say you wanted to check whether the mouse is over a 3D object? You can’t make that part of the UI and need to do 3D raycasts

how do i do that

I assume you mean [2.]?

For that you can use the ray-casting system in Godot: https://docs.godotengine.org/en/stable/tutorials/physics/ray-casting.html

The tutorial I shared previously also gives a really good explanation of the topic: https://www.youtube.com/watch?v=mmvIkkKJVlQ