Performance considerations for stand alone XR

Help XR
1

Seeing this came up on the Discord a few days ago, figured I’d put the info we have so far here.

Most stand alone XR devices today are running Android often with Qualcomm GPUs like the XR2 chipset (Quest, PICO 4, HTC Elite, etc.). While very impressive hardware their architecture and constraints due to the mobile format mean that there are strict limits to what these GPUs can do. A lot of techniques Desktop GPUs have no difficulty with, these mobile GPUs will struggle with.

Below is a list of suggestions and considerations to be aware of when making XR applications for these platforms.

Note: this is just a start, hopefully people will add many suggestions as reactions and I’ll try and update this top post with new information as time progresses.

  1. Do use the compatibility renderer. While more limited in capabilities, its simplicity works well on stand alone devices. You can use the Mobile renderer, especially in the upcoming 4.3 release but there is still work to bring it in line with the compatibility renderer.
  2. Do not use screen space effects such as glow, DOF, etc. Be aware that Godot will auto enable glow, you must turn it off. Not only do mobile GPUs struggle with these effects, they also turn off various optimisations.
  3. Do use foveated rendering. For the compatibility renderer check the OpenXR settings and set Foveation level to high and enable Foveation dynamic. For the Mobile renderer enable VRS on your main viewport by adding the following line in your startup code: get_viewport().vrs_mode = Viewport.VRS_XR (note: will only be active on devices that support this).
  4. (4.3+) Do use MSAA x2, it’s practically free on mobile GPUs.
  5. Do not read from screen texture or depth texture nor enable effects like refraction on materials. These are absolute performance killers on mobile GPUs.
  6. Do optimise your materials, this is something you will want to do late in your development cycle and only if you discover bottlenecks but converting your materials to shader materials and removing unwanted texture lookups, Godot materials are made for convenience and will by default have overhead and can often be simplified. Converting to shader material is also a great way to learn about Godot shader code and getting a feel for what are costly shader options.
  7. Do limit texture usage per material, it’s tempting to set that roughness, and metallic, and normal, and AO and depth texture, but mobile GPUs struggle when reading from multiple textures. If you can, combine them (roughness, metallic and AO can easily be one texture by using each color channel for each texture).
  8. Avoid UV calculations in the fragment shader, calculate your UVs in the vertex shader and use varying variables for your texture lookups.
  9. Avoid lights and especially shadows. They are very expensive on mobile hardware. Bake lighting if you can, 4.3 is getting some really good improvements for this. If your player has a flashlight, use a single spotlight. Use omni lights only with static shadow buffers.

Note: A clarification on point 4, MSAAx2 is practically free on mobile GPUs due to the unique nature of tile based GPUs but only under heavy restrictions. E.g. not enabling any effects that require Godot to perform multiple render passes. These effects have already been named as don’ts but they are extra don’ts when combined with MSAA. Enabling effects such as: glow/bloom, scaling and certain color corrections will have a exponential performance degradation that can easily reduce your framerate by over 30%.

7 Likes
2

For those who want to know how to combine textures, you can see me struggle here on screen with Krita (I’m not a graphics artist):
https://www.youtube.com/live/8d7TM6f14ps?si=Roq65WFe2gw8ln_v&t=5112

Note that this stream is a good watch on the subject of optimising shaders as I do several.

1 Like
3

You can use the ImageMagick command listed in https://github.com/godotengine/godot-proposals/issues/2316:

convert ao.png roughness.png metallic.png -combine orm.png

(On systems that don’t have legacy aliases installed, use magick convert instead.)

Also, to save memory and file size, you can use an half-resolution texture for the ORM map as it tends to feature lower-frequency information than the base color and normal maps. Doing so reduces memory usage for the ORM texture by 75%.

Also, expensive material features like height mapping should always be disabled when targeting XR.

1 Like
4

Great stuff, this should be pinned in the XR category if that’s possible.

2 Likes
5

Excellent! This is what I have been searching for. My biggest struggle was wondering why VR looked so bad in my Quest 3 headset.

1 Like
6

Yes it looks bad on quest 3 compared to quest 2.

7

Considering Samsung is releasing an eye tracking capable, standalone VR device in the next year, and other companies also having announced them:

Is it already planned, to add eye tracking based Dynamic Foveated Rendering and as well Dynamic VRS?

8

@shalokshalom this is a bit of a difficult one to answer because its a little bit of yes and a little bit of no.

The compatibility renderer implements foveated rendering through an OpenXR Meta vendor extensions that is only supported on some headsets. OpenXR doesn’t (yet) have an alternative and it’s unknown at this point in time whether Android XR implements the Meta extension, is introducing their own vendor extension or if there is another scenario not yet publicised.

On the Vulkan side we have implemented our own version of VRS for foveated rendering. We thus have fixed foveated rendering covered however for eye tracked foveated rendering currently OpenXR does not expose a usable extension that provides good information on where the eye centers are.
We have an experimental solution using the eye gaze extension but its not really meant or suitable for this.
Also there is an issue that there are actually two competing VRS implementations on Vulkan.

We currently support the Fragment Rate Shading extension which is the Khronos core extension with all bells and whistles, but its badly supported on standalone hardware.
And we’ve got support for the Fragment Density Map extension incoming in Godot 4.4 (hopefully), which is the extension that gives the best VRS performance boost on Qualcomm XR2 hardware.

The bottom line though is, until a little more is exposed through OpenXR, eye tracked foveated rendering might not be immediately available. It all depends on which current extensions Google will be supporting.

1 Like