Multithreading intersect_ray() / raycasts

Help
,
1

Godot Version

4.2.1

Question

Hi hi!
I’m doing some tests with multi-threading. I’ve got a use case in which I want to deal with many thousands of batched raycasts, I’ve got a lot of leeway with how fast those should be executed so doing those on a separate thread makes a lot of sense to me.

I’ve made a quick prototype script in GDScript to sort things out since I’m new to multi-threading. My code seems to work fine, I’m able to execute many many thousands of raycasts on a single worker thread, maintaining 60 fps.

I verify my ray-cast results by drawing debug spheres on hit-positions, and lines on no hits, which works almost entirely as desired, very few raycasts give me this error:

MultithreadedRaycaster.gd:41 @ ray_cast(): Condition "space->locked" is true. Returning: false

According to the docs:

… Due to this, the only time accessing space is safe is during the Node._physics_process() callback. Accessing it from outside this function may result in an error due to space being locked.
Ray-casting — Godot Engine (stable) documentation in English

It is explained why this error happens, but not what locked exactly means or entails.
Does this mean it simply isn’t able to execute the raycast, or does it mean the space state information is outdated but is still able to execute the intersect_ray function (Which is what it looks like)?

Global Scope singletons are all thread-safe. Accessing servers from threads is supported (for RenderingServer and Physics servers, ensure threaded or thread-safe operation is enabled in the project settings!).
Thread-safe APIs — Godot Engine (stable) documentation in English

I cannot find any setting related to “threaded or thread-safe operation”. I don’t know if this is deprecated because this page has not been updated yet for 4.2, but it seems like this might alleviate the issue.

Here are my results with 15,000 raycasts, in a test scene with my node in a closed room. All hit results should return positive. If I get a negative hit result, I draw a red line showing to display that failed raycast.

extends Node3D

var worker_thread_1 : Thread
var directions : Array

func _ready():
	directions = generate_directions(15000)
	worker_thread_1 = Thread.new()

func _physics_process(delta):
	global_position = global_position + Vector3(0,0.002,0)
	
	DebugDraw3D.draw_sphere(global_position, 0.02, Color.PURPLE, delta)
	
	# start thread when it's not doing anything
	if worker_thread_1.is_started() == false:
		# Grabbing space state on main thread, sending it into raycast function
		var space_state = get_world_3d().direct_space_state
		worker_thread_1.start(raycast_to_directions.bind(directions, global_position, space_state))
	
	# stop thread when it's done
	if worker_thread_1.is_alive() == false:
		worker_thread_1.wait_to_finish()

Would love to hear if anyone has any ideas or could help me out here. Thanks!

2

Locked means that it’s iterating and doing work and you should not trust the result. It may work or it may not work depending on where in the process the server is. You should avoid querying the server at that moment.

You can enable both the 2D and 3D physics servers to run on separated threads by enabling Run on Separated Thread in the project settings under Physics / 2D and Physics / 3D. You’ll need to enable Advanced Settings to be able to change this setting.

3

Locked means that it’s iterating and doing work and you should not trust the result. It may work or it may not work depending on where in the process the server is. You should avoid querying the server at that moment.

Thanks for the explanation, that makes sense. Do you know of a way to check whether or not a space is locked? Or of any particular way of ensuring my intersect_ray function call isn’t done when the physics server is busy?

You can enable both the 2D and 3D physics servers to run on separated threads by enabling Run on Separated Thread in the project settings under Physics / 2D and Physics / 3D. You’ll need to enable Advanced Settings to be able to change this setting.

That doesn’t seem like the same setting mentioned in the documentation. “Run on separate thread” means the entire physics server is running on a different thread than the main thread. The settings mentioned in the documentation “Threaded or Thread-safe operation” seem to insinuate something, at least it reads like that to me.

4

It’s locked when it’s outside _physics_process, only then can you do these things

5

Would a proposal to be able to retrieve a bool from the physics server that states whether or not space is locked at the moment or not make sense, or is there a straightforward issue here that I’m not seeing?

It seems like intersect_ray() works in the vast majority of calls on the separate thread, and the documentation states:

Accessing it from outside this function may result in an error due to space being locked.

I can bump the amount of intersect_ray() calls on a single thread to 250,000 with the vast majority returning the expected result, and still have the game run at 60 FPS.
The ability to pre-validate whether or not an intersect_ray call is possible at a given time seems like it could open up a lot of possibilities.

6
7

I stumbled upon that post as well, unfortunately that resulted in the function barely, or never, getting called.
It seems like it is possible for the space to be unlocked and accessible while the engine is not in a physics-frame.

8

You can use many threads but you can not use them async.

They need to finish in the same physics process step so the physics space can unlock again.

This means when the physics process starts you can throw a group task with x-amount of raycast at the WorkerThreadPool and immediatly wait to finished for the group task.

This assures that all threads are finished at the end of the physics process call so the space can unlock and the PhysicServer can finish its sync.

The engine does the very same internally for a lot of servers, creates a threadpool group task and waits to finished immediately in the physics process callback.

2 Likes
9

Thanks for the explanation, that makes a lot of sense!

Adding an individual task to the WorkerThreadPool seems to work fine, however the ray-casts offloaded to a single different thread doesn’t seem to provide any performance improvement. Adding multiple tasks, or using group tasks, seems to give me the same issues as executing the raycasts on async threads despite waiting to finish those tasks in the physics process.
Despite getting those issues, I’m not getting any errors stating the space is locked.