Viewport issues with parse_input_event,push_input

Godot Version

4.6.2stable on quest3

Question

I have a viewport under an XR composition layer setup exactly like the example in the documentation for composition layers. Which works fine for certain types of input but I want to add scrolling by pressing up or down on the controller. No matter if I use push_input or parse_input_event after I use scroll the first time any UI will no longer work except for the very first control node. No matter where I pull the trigger in the UI only the first node will grab focus. Does not matter if the node i'm pulling the trigger on has stop input or anything.

Drag and drop on treeitems doesn’t work either, but that seems like a separate issue since input works as normal.