Does Eye Tracking work on Pico 4E?

jbizlv · June 13, 2024, 8:31pm

Godot Version

4.2.2

Question

Does Eye tracking work on the Pico 4E?

I see that in Oct. of 23 it was being discussed as being added separately.

github.com/GodotVR/godot_openxr_vendors

Add Meta configs for the eye gaze interaction extension

GodotVR:master ← m4gr3d:add_eye_gaze_interaction_configs

opened 05:20AM - 01 Oct 23 UTC

m4gr3d

+150 -23

The first commit is a small change to address and fix https://github.com/GodotVR…/godot_openxr_loaders/issues/46 The second commit contains the majority of the changes and is the counterpart to https://github.com/godotengine/godot/pull/82614: - Adds the necessary config option to enable eye tracking for Meta devices - Defines `XR_EXT_eye_gaze_interaction` as a feature tag to set when eye tracking is enabled - On mobile xr devices, this feature tag is used to gate the eye gaze interaction extension - Adds the necessary Meta permissions when eye tracking is enabled - Updates the demo project to showcase eye tracking (tested on Quest Pro device) - Updates Meta's `GodotPlugin` instance to return `true` for the `PERMISSION_XR_EXT_eye_gaze_interaction` feature tag when the necessary permissions are granted.

I have a Pico 4E coming tomorrow and would like to start developing with it as soon as possible.

If it is not added and it needs testing I’m available for testing starting tomorrow.

jbizlv · June 13, 2024, 9:56pm

I just learned that the Pico 4E & Pico 4 Pro Eye Tracking does not work with PCVR / OpenXR which is how Godot interfaces with it so I’m guessing this won’t work.

soapspangledgames · June 13, 2024, 10:11pm

OpenXR seems to still have significant issues.

jbizlv · June 13, 2024, 10:32pm

Thanks for the input. That’s unfortunate. I purchased a Pico 4E because a client has one and needs me to develop some apps with Eye Tracking for it. I even asked the seller if it worked for development purposes on PC and they said no problem.

I guess I have to dev on Meta Quest Pro and hope it works on Pico when I export… we’ll see!

If it doesn’t work, he may be getting a quest pro sooner than he expected. He’s not opposed to it, I’m just trying to use what the client has.

jbizlv · June 13, 2024, 11:47pm

So after further research. It seems Quest Pro doesn’t support XR_EXT_eye_gaze_interaction out of the box either. It’s not just a Pico issue.

But it looks like you can install OpenXR Toolkit found on github and use Virtual Desktop XR on Quest Pro to get the input from XR_EXT_eye_gaze_interaction.

Is this the only way to get Eye Tracking to work on the Quest Pro in Godot?

If anyone has any experience with this I’d appreciate some feedback.

astrofra · November 29, 2024, 9:47am

Hi,
from my experience, Meta decided not to use the generic OpenXR extensions for hand, eyes and face tracking. This doesn’t make any sense but it is as it is.

I checked in our own code and we’re using

XR_FB_EYE_TRACKING_SOCIAL_EXTENSION_NAME
XR_FB_FACE_TRACKING2_EXTENSION_NAME

On a Meta Quest Pro.

I wouldn’t recommend to use a Meta headset for profesionnal use case, though. Our experience with this headset is rather … complicated. Expecially the headset tracking that isn’t stable when you have to use it a whole day long, and the fact that you can’t control nor rollback the software updates.

We’re currently evaluating the Pico 3 Pro Eye and it’s much more stable and usable so far… to be continued.

It’s unfortunate you couldn’t find the support of eye tracking in Godot for Pico 3 Pro Eye … let’s hope things will move on

Good luck on your project.