Handling Input for 3D GUI with a Constrained Mouse

Godot 4.2.2

I’ve got a wide-reaching question motivated by a specific problem.
How do I go about processing mouse movement and clicks on a 3D GUI area while using Input.MOUSE_MODE_CAPTURED?

I’m currently working on a 3D, first-person project where I have the desire to make a menu that exists as an object in the world. I have said menu, as a SubViewport displayed on a MeshInstance3D(QuadMesh) with an Area3D handling collisions and mouse related interactions.

After some research, I came along this video (https://youtu.be/Hih8PD2xeMw?si=vNEBeyZepYCFlLoX) by Chevifier and the “GUI in 3D Viewport Demo” resource on the Godot Asset Library Projects page. These implementations (which are very similar) work very well in any case using Input.MOUSE_MODE_VISIBLE. However, they do not work for the case of using Input.MOUSE_MODE_CAPTURED which is desirable for first-person camera control.

After many hours of searching, I have found little documentation on why this would be the case, however, there is a video (https://youtu.be/qEjxZc6TAM8?si=gh3K_19TgqOM4Kdw) by MTZD that quickly (in a matter of 25 seconds) outlines his process of getting it to work using raycasts (a concept that makes sense, given the versatility and common usage of raycasts as a method of handling line of sight and input on an object in a space.) The problem lies in that this implementation is in C#, and when I tried converting the ideas to GDscript, I hit a wall.

So below I have included my attempt at porting MTZD’s work into GDscript, but more than just explaining why my work is wrong, I would be very appreciative if someone could explain the process of how to work around the issue of constrained mouse input not registering for 3D GUI objects, both so that I can implement those ideas in my own project, and so that others who want to do something similar have a resource to come to when they inevitably reach the same issue.

The Script:

extends Node3D

var range = 2
# Used for checking if the mouse is inside the Area3D.
var is_mouse_inside = false
# The last processed input touch/mouse event. To calculate relative movement.
var last_event_pos2D = null
# The time of the last event in seconds since engine start.
var last_event_time: float = -1.0

@onready var node_viewport = $SubViewport
@onready var node_quad = $"../Level_Assets/GUI_Screen"
@onready var node_area = $"../Level_Assets/GUI_Screen/Area3D"

func _ready():

func _mouse_entered_area():
	is_mouse_inside = true

func _mouse_exited_area():
	is_mouse_inside = false

func _unhandled_input(event):
	# Check if the event is a non-mouse/non-touch event
	for mouse_event in [InputEventMouseButton, InputEventMouseMotion, InputEventScreenDrag, InputEventScreenTouch]:
		if is_instance_of(event, mouse_event):
			# If the event is a mouse/touch event, then we can ignore it here, because it will be
			# handled via Physics Picking.

func HandleMouse(_camera: Camera3D, event: InputEvent, event_position: Vector3, _normal: Vector3, _shape_idx: int):
	is_mouse_inside = FindMouse(event_position)
	_mouse_input_event(event, event_position)
func _mouse_input_event(event: InputEvent, event_position: Vector3):
	# Get mesh size to detect edges and make conversions. This code only support PlaneMesh and QuadMesh.
	var quad_mesh_size = node_quad.mesh.size

	# Event position in Area3D in world coordinate space.
	var event_pos3D = event_position

	# Current time in seconds since engine start.
	var now: float = Time.get_ticks_msec() / 1000.0

	# Convert position to a coordinate space relative to the Area3D node.
	# NOTE: affine_inverse accounts for the Area3D node's scale, rotation, and position in the scene!
	event_pos3D = node_quad.global_transform.affine_inverse() * event_pos3D

	var event_pos2D: Vector2 = Vector2()

	if is_mouse_inside:
		# Convert the relative event position from 3D to 2D.
		event_pos2D = Vector2(event_pos3D.x, -event_pos3D.y)

		# Right now the event position's range is the following: (-quad_size/2) -> (quad_size/2)
		# We need to convert it into the following range: -0.5 -> 0.5
		event_pos2D.x = event_pos2D.x / quad_mesh_size.x
		event_pos2D.y = event_pos2D.y / quad_mesh_size.y
		# Then we need to convert it into the following range: 0 -> 1
		event_pos2D.x += 0.5
		event_pos2D.y += 0.5

		# Finally, we convert the position to the following range: 0 -> viewport.size
		event_pos2D.x *= node_viewport.size.x
		event_pos2D.y *= node_viewport.size.y
		# We need to do these conversions so the event's position is in the viewport's coordinate system.

	elif last_event_pos2D != null:
		# Fall back to the last known event position.
		event_pos2D = last_event_pos2D

	# Set the event's position and global position.
	event.position = event_pos2D
	if event is InputEventMouse:
		event.global_position = event_pos2D

	# Calculate the relative event distance.
	if event is InputEventMouseMotion or event is InputEventScreenDrag:
		# If there is not a stored previous position, then we'll assume there is no relative motion.
		if last_event_pos2D == null:
			event.relative = Vector2(0, 0)
		# If there is a stored previous position, then we'll calculate the relative position by subtracting
		# the previous position from the new position. This will give us the distance the event traveled from prev_pos.
			event.relative = event_pos2D - last_event_pos2D
			event.velocity = event.relative / (now - last_event_time)

	# Update last_event_pos2D with the position we just calculated.
	last_event_pos2D = event_pos2D

	# Update last_event_time to current time.
	last_event_time = now

	# Finally, send the processed input event to the viewport.

func FindMouse(globalPosition: Vector3) -> bool:
	var center = get_viewport().size / 2 
	var origin = $"../Character/Head/Camera".project_ray_origin(center)
	var end = origin + ($"../Character/Head/Camera".project_ray_normal(center)) * range
	var RayCast = PhysicsRayQueryParameters3D.create(origin, end)
	var Hit = $"../Character/Head/Camera".get_world_3d().direct_space_state.intersect_ray(RayCast)
	var position = Vector3(0,0,0)
	if not Hit.is_empty():
		position = Hit.get("position")
		return true
		return false