UI breaks on one viewport while switching back from another viewport

Godot Version

Godot Engine v4.3.stable.steam.77dcf97d8

Question

I have a game similar to Welcome to the Game games where you are either at the PC and the display covers your screen fully and cursor is visble or you’re not at the PC and have FPP controls enabled to move around. To separate both of these environments I use two separate viewports - one for the player body and one for the display of the PC. I needed to do this since UI is tied to a viewport and obviously I don’t want to have the PC UI in the eyes of my player once he gets off.

So I got a code below that is ran by certain events:

if is_on_cameras:
	Input.set_mouse_mode(Input.MOUSE_MODE_CONFINED)
	camera_viewport_container.visible = true
	fpp_viewport_container.visible = false
	player_body.set_process_input(false)
else:
	Input.set_mouse_mode(Input.MOUSE_MODE_CAPTURED)
	camera_viewport_container.visible = false
	fpp_viewport_container.visible = true
	player_body.set_process_input(true)

Now for some reason when you toggle the viewports like this some funny stuff with UI begins to happen. It seems that focus is desynchronized, like pressing a button acts like pressing a button that was last in focus before running the above code, or the button just doesn’t work at all, though hover logic still works as expected.

Does anybody have an idea what can be going on?