Eye-tracker support for HTC Vive Pro Eye

I tried, but unfortunately I’m not able to modify the original post anymore.

1 Like

Thanks for trying, maybe @winston-yallow can help out

moved this to the XR category! :sparkles:

1 Like

In Godot 4.2.2, I have everything configured per the instructions for the Vive Eye as @mux213 has supplied above. Everything looks like it’s configured correctly.

image
image

When I run the app in godot I get the following output:

I have added the is_eye_gaze_interaction_supported() == true to the output. This is the result of calling:

if xr_interface.is_eye_gaze_interaction_supported():
print(“is_eye_gaze_interaction_supported() == true”)

So Godot 4.2.2 sees that eye gaze interaction is supported… However I get this just before I’m told gaze interaction is supported:
OpenXR: failed to create action eye_gaze ! [ XR_ERROR_PATH_UNSUPPORTED ]

So I’m guessing my action_map setup is not correct, or the tracker is incorrect. I say the tracker may be incorrect because of an error I received in verbose output mode.

I have enabled verbose logging in project settings and noticed this line:
OpenXR: Interaction profile for /user/eyes_ext changed to /interaction_profiles/ext/eye_gaze_interaction

So I changed my Tracker to be:


(edit) ^ I know this is not correct and is pointing to the wrong place, but it’s the only way the error goes away

I no longer get this error:
OpenXR: failed to create action eye_gaze ! [ XR_ERROR_PATH_UNSUPPORTED ]

However when I output the transform of the eye_gaze tracker:
@onready var eye_gaze = $XROrigin3D/EyeGaze

func _process(_delta):
print("eye_gaze: ", eye_gaze.transform)

The transform is always:
eye_gaze: [X: (1, 0, 0), Y: (0, 1, 0), Z: (0, 0, 1), O: (0, 0, 0)]

Am I not grabbing the eye tracker data from the eye_gaze object correctly?
i.e.
print("eye_gaze: ", eye_gaze.transform)

@jbizlv note the error XR_ERROR_PATH_UNSUPPORTED when it attempts to create the eye_gaze action. That is your issue.

I’ve run into this a few times now, there are certain XR runtimes that fall over the use of an underscore for SOME extensions. This goes against the OpenXR standard but c’est la vie. Try renaming the action to eyegaze and change that on your tracker, see if it works.

@mux213 I have changed the action map & tracker to eyegaze:
image
image

However I still get the same error:

OpenXR: failed to create action eyegaze ! [ XR_ERROR_PATH_UNSUPPORTED ]

I’m using this EyeTracker:

And this OpenXR Tools Runtime:

If you know of others that do work for Quest Pro, please let me know & I’ll give them a shot.

Hmm, weird, I’m kinda out of ideas. I am assuming you are trying to make this work on PC over Meta link (as the eye gaze interaction works natively on Quest Pro)? You may want to leave an issue on Matthieus repo. It’s possible his solution only works with Metas own XR runtime or over SteamVR, but I don’t have any direct experience with it.

@mux213

Yes I am using PC over Meta Link via USBC.

I have tried using pascal case, since I see some docs for Unity using these tools, and the error goes away… But the tracker still reports nothing:

image
image

Lets assume I do get everything configured correctly… will I get the data from the transform or position of my eye_gaze object in order to grab where the user is looking?

@onready var eye_gaze = $XROrigin3D/EyeGaze

func _process(_delta):
print("eye_gaze: ", eye_gaze .transform)
print("eye_gaze: ", eye_gaze .position)

That should work, it should place the XRNode3D right between the eyes and then orient it correctly.

Note that the position is just the origin of the transform, they go hand in hand.
But it is then transform.basis.z that gives you the direction you’re looking at.

You also likely want to use global_transform depending on what it is you’re trying to do.

What I often do to test is add a MeshInstance3D object as a child of the XRNode3D, and set it’s z position to something like 2.0 or 3.0 and give it a small sphere mesh (z might need to be -2.0 or -3.0) .
That mesh should then follow where I’m looking.

Gives you a good idea of what is going on.

1 Like

Right now my goal is to just grab where the user is looking so I know what/where they are looking at on the screen. I’ll then use this to make a selection.

I’m not really moving anything on the screen, yet. It’s more to collect information & make determinations/selections based on what they are looking at.

I will probably be moving a cursor at some point in the near future once I get it functional.

Thanks for all your help in trying to get the Quest Pro working for me.

1 Like

I have not been able to get Godot to communicate with a Quest Pro’s eye tracking through PCVR. I believe there is an issue with Mathhieus OpenXR Toolset for SteamVR

I have confirmed that the configs from the Vive Eye will work when running my app native on Android.

I was able to grab eye gaze data and output it to a debug window within VR while running native on Android.

Hopefully Matthieu has some time to address this issue. He’s done some amazing work! Would be game changing if Godot could get native Quest Pro PCVR Eye Tacking support!

1 Like

Dear @jbizlv, @mux213 ,

I am working on an eye-gaze tracking solution with Oculus Quest Pro on PC and streaming the image with SteamLink. Did you find any solution to the error: OpenXR: failed to create action eyegaze ! [ XR_ERROR_PATH_UNSUPPORTED ] ??

Thank you for your comments in advance!

I did not. I have not abondoned my project, but other projects have come up & I hve put this project on the back burner.

If you find a solution please post it. I would ppreciate it.

I have (unsuccessfully) been trying to do the same as @jbizlv for a week now, only receving the error “XR_ERROR_PATH_UNSUPPORTED” no matter if I use ‘eye_gaze’ or ‘eyegaze’ as the name in the actionmap. I am using Godot version 4.3.

Using the testing tool by Matthieu (Home · mbucchia/OpenXR-Eye-Trackers Wiki · GitHub), I can confirm that my computer is receiving (and can use) eye tracking data from Meta Quest Pro over Meta Quest Link (via Air Link).

If anyone manages to make this work with Meta Quest Pro, please share! I think a working Godot example project uploaded to Github would be very useful for others to troubleshoot. I will upload any solution if I manage to find one, and share it here.

Problem solved!

Use Steam Link / SteamVR instead of Meta Link.
In addition to the mbucchia’s guidance here, make sure to set the option ‘Meta Plugin Compatibility’ to ‘On’ in SteamVR Settings under the ‘OpenXR’ page. See following image for more clarity:

For anyone wondering, that ‘1 active’-tag on ‘Manage OpenXR API Layers’ just denotes that I have the ‘OpenXR toolkit API layer’ turned on.

I made a miniscule Godot [4.3] project for anyone wanting to verify that Godot is able to receive eye tracking data from their VR setup. It can be found here.

1 Like