Custom UDP packages alongside scene replication - best approach? (without rpc)

Godot Version

Godot 4.6

Question

In c++ I want to send unreliable UDP data from client to server and avoid the overhead of rpc calls, but I am not sure what the best approach is, while playing nice with built in scene replication.

I have tried put_packet/get_package, but there does not seem to be a way to identify or extract custom data in stream that is also used by scene replication.

Do I need to make a separate connection (e.g. PacketPeerUDP) or is there a way to use existing UDP stream?