Multithreading intersect_ray() / raycasts

[lander-vr]

Godot Version

4.2.1

Question

Hi hi!
I’m doing some tests with multi-threading. I’ve got a use case in which I want to deal with many thousands of batched raycasts, I’ve got a lot of leeway with how fast those should be executed so doing those on a separate thread makes a lot of sense to me.

I’ve made a quick prototype script in GDScript to sort things out since I’m new to multi-threading. My code seems to work fine, I’m able to execute many many thousands of raycasts on a single worker thread, maintaining 60 fps.

I verify my ray-cast results by drawing debug spheres on hit-positions, and lines on no hits, which works almost entirely as desired, very few raycasts give me this error:

MultithreadedRaycaster.gd:41 @ ray_cast(): Condition "space->locked" is true. Returning: false

According to the docs:

… Due to this, the only time accessing space is safe is during the Node._physics_process() callback. Accessing it from outside this function may result in an error due to space being locked.
Ray-casting — Godot Engine (stable) documentation in English

It is explained why this error happens, but not what locked exactly means or entails.
Does this mean it simply isn’t able to execute the raycast, or does it mean the space state information is outdated but is still able to execute the intersect_ray function (Which is what it looks like)?

Global Scope singletons are all thread-safe. Accessing servers from threads is supported (for RenderingServer and Physics servers, ensure threaded or thread-safe operation is enabled in the project settings!).
Thread-safe APIs — Godot Engine (stable) documentation in English

I cannot find any setting related to “threaded or thread-safe operation”. I don’t know if this is deprecated because this page has not been updated yet for 4.2, but it seems like this might alleviate the issue.

Here are my results with 15,000 raycasts, in a test scene with my node in a closed room. All hit results should return positive. If I get a negative hit result, I draw a red line showing to display that failed raycast.

extends Node3D

var worker_thread_1 : Thread
var directions : Array

func _ready():
	directions = generate_directions(15000)
	worker_thread_1 = Thread.new()

func _physics_process(delta):
	global_position = global_position + Vector3(0,0.002,0)
	
	DebugDraw3D.draw_sphere(global_position, 0.02, Color.PURPLE, delta)
	
	# start thread when it's not doing anything
	if worker_thread_1.is_started() == false:
		# Grabbing space state on main thread, sending it into raycast function
		var space_state = get_world_3d().direct_space_state
		worker_thread_1.start(raycast_to_directions.bind(directions, global_position, space_state))
	
	# stop thread when it's done
	if worker_thread_1.is_alive() == false:
		worker_thread_1.wait_to_finish()

Would love to hear if anyone has any ideas or could help me out here. Thanks!

[mrcdk]

Locked means that it’s iterating and doing work and you should not trust the result. It may work or it may not work depending on where in the process the server is. You should avoid querying the server at that moment.

You can enable both the 2D and 3D physics servers to run on separated threads by enabling Run on Separated Thread in the project settings under Physics / 2D and Physics / 3D. You’ll need to enable Advanced Settings to be able to change this setting.

[lander-vr]

Locked means that it’s iterating and doing work and you should not trust the result. It may work or it may not work depending on where in the process the server is. You should avoid querying the server at that moment.

Thanks for the explanation, that makes sense. Do you know of a way to check whether or not a space is locked? Or of any particular way of ensuring my intersect_ray function call isn’t done when the physics server is busy?

You can enable both the 2D and 3D physics servers to run on separated threads by enabling Run on Separated Thread in the project settings under Physics / 2D and Physics / 3D. You’ll need to enable Advanced Settings to be able to change this setting.

That doesn’t seem like the same setting mentioned in the documentation. “Run on separate thread” means the entire physics server is running on a different thread than the main thread. The settings mentioned in the documentation “Threaded or Thread-safe operation” seem to insinuate something, at least it reads like that to me.

[athousandships]

It’s locked when it’s outside _physics_process, only then can you do these things

[lander-vr]

Would a proposal to be able to retrieve a bool from the physics server that states whether or not space is locked at the moment or not make sense, or is there a straightforward issue here that I’m not seeing?

It seems like intersect_ray() works in the vast majority of calls on the separate thread, and the documentation states:

Accessing it from outside this function may result in an error due to space being locked.

I can bump the amount of intersect_ray() calls on a single thread to 250,000 with the vast majority returning the expected result, and still have the game run at 60 FPS.
The ability to pre-validate whether or not an intersect_ray call is possible at a given time seems like it could open up a lot of possibilities.

[athousandships]

When can the physics space be locked?

[lander-vr]

I stumbled upon that post as well, unfortunately that resulted in the function barely, or never, getting called.
It seems like it is possible for the space to be unlocked and accessible while the engine is not in a physics-frame.

[smix8]

You can use many threads but you can not use them async.

They need to finish in the same physics process step so the physics space can unlock again.

This means when the physics process starts you can throw a group task with x-amount of raycast at the WorkerThreadPool and immediatly wait to finished for the group task.

This assures that all threads are finished at the end of the physics process call so the space can unlock and the PhysicServer can finish its sync.

The engine does the very same internally for a lot of servers, creates a threadpool group task and waits to finished immediately in the physics process callback.

[lander-vr]

Thanks for the explanation, that makes a lot of sense!

Adding an individual task to the WorkerThreadPool seems to work fine, however the ray-casts offloaded to a single different thread doesn’t seem to provide any performance improvement. Adding multiple tasks, or using group tasks, seems to give me the same issues as executing the raycasts on async threads despite waiting to finish those tasks in the physics process.
Despite getting those issues, I’m not getting any errors stating the space is locked.

[IntrepidMarmot]

I am having the same issue with workerthreads group. did you ever figure out what is going on?

[pennyloafers]

@lander-vr

locked exactly means or entails

Mutex/semaphore locked, its a common practice in multi-threading to restrict data manipulation on shared memory, so that data updates can spread to the other core’s cache memory that may be out dated. If you dont lock when reading/writing shared memory you may be in for undefined behavior.

Generally its safe to read from multiple threads simultaneously, but if a write happens you need to be careful, as some of the read threads may have stale data.

In this case you shouldn’t run raycast threads when the space is locked because some collision objects maybe getting deleted and you wouldn’t want to return a dangling reference to a deleted collison body.

A general google search on thread safety, mutex and cache coherency can give you more answers as this is a fundamental concept of computer science.

[pennyloafers]

Depending on your goal I think you could probably create a compute shader to do this faster and with more rays, and not rely on the physics engine.

[lander-vr]

What @pennyloafers said is bang on.

While the physics space state is pretty stable overall, there’s no actual assurance that it’s representative of the scene outside the physics thread. Which is why, despite it mostly working in the video I shared, it sometimes returns incorrect results (the ray misses). Those ray misses are likely happening at the same moment the space-state is being updated.

With this many rays, raytracing on a compute shader definitely makes more sense. Here’s a relevant PR doing the groundwork for raytracing: Vulkan raytracing plumbing by Fahien · Pull Request #99119 · godotengine/godot · GitHub