Timing: non-game (audio-)visual app with eye-tracking

General
1

Hi,

Sorry if this question sounds confused - I know very little about game engines and Godot. I know a fair bit about frame by frame visual display, based on low-level openGL APIs, wrappers around Pyglet, etc.

I am trying to find out whether Godot could be a good solution to create a program that a) allow frame-by-frame control with precise-timing of fairly simple visual stimuli, b) has a GUI-based user-interface to select the parameters of the stimulation and c) interact with an eye-tracker device that is either a hardware device communicating via USB/ethernet or is a machine-learning model running on input from a webcam. This “app” could be for desktops, but potentially also for tablets/mobile phones.

From what I have read, Godot can be used to make a GUI interface, and it seems that precise-timing of fairly simple visual displays (without frame loss etc, and with changes happening on the frame that is closest to the supplied time) should be possible with a good GPU ? And it would also be possible to calculate the timing desired based on the inter-frame interval so that the timing is frame-based ? Is it “easy” to interface with new streams of say eye-position x,y data coming in via the USB/ethernet or from a machine-learning model running on the same computer and then use the eye-position to control the visual stimulation ? Alternatively, would it be possible to output timing markers from Godot (about when a change happened) so that it can be saved along with the eye-tracking data by another process ?

The alternative would be to use Pyglet or Pygame and handle everything in Python - is that just a better choice if complex game elements are not involved ?

Thank you so much, and happy to clarify of course if I am being unclear somewhere.

2

I have now learnt quite a bit myself talking with GPT-4 and looking through the docs. I learnt that Godot can communicate in many ways with a python process on the same computer or another server through sockets, inter-process communication, http, etc and this python process can handle the eye-tracking. Also, the Godot shader seems to work quite well for 2D stimuli, and can handle the user-interface GUI as well. Please let me know if and how I am wrong here.

I did have one specific question: is there a straightforward way for Godot to read data via USB on the same computer ? It seems this can be done via a C# library from within Godot itself - is this correct ?