Can I render some objects differently to a secondary buffer for OBS Virtual Camera?

Godot Version

4.2.1

Question

Can I render some objects differently to a secondary buffer for OBS Virtual Camera?

Quick context for you: I’m exploring if I can recreate a project that was a 2D card game “goldfishing” application (tabletop object simulation and info tracking, but no rule enforcement). One of its most attractive features was that it could optionally perform rendering to an additional render target and pass the address of that render target’s buffer to a separate DLL that turned that into the output stream for the OBS Virtual Camera device. Therefore, I’m trying to prototype if I can accomplish this in Godot.

For example, the cards in one’s hand were displayed along the bottom edge of the screen in an invisible tray. The normal render that the user saw showed the actual card faces, but in the secondary render for virtual camera output they would only be rendered with the card back sprite in order to hide information from opponents. Cards in play, as public information, were rendered normally in both.

The previous project had low-level drawing events, though, where switching draw target and the sprite drawing commands were all explicitly coded.

In looking around the forums a little bit, I came upon a thread mentioning that this aspect of 3D shaders solved their issue: Implement CAMERA_VISIBLE_LAYERS as built-in shader variable - PR #67387. On first look, this might be a good lead toward what I want to achieve.

If this is only possible with 3D and not 2D, then I don’t mind switching to work in 3D instead, but if there is a simpler way to accomplish this in 2D, then I’m all ears.

Edit: Neglected to mention that I’m fine with this requiring the C#-supporting version and sticking to desktop due to that.

I decided, since I’m solving this piece-by-piece, to record the steps of my solution.

Alternate Rendering w/ Selective Difference

The whole purpose of rendering certain cards differently in another render target is that some zones are private information to the player (such as the cards in their hand), and some zones are public information to all players.

I used a Viewport’s Canvas Cull Mask and named Layer 1 as ‘Private’ and Layer 2 as ‘Public’, and its world_2d is set to the main window’s world_2d. I started with a separate non-embedded Window as the Viewport just so it was easier to see what was going on during development. It’s just the same size as the original window, but with the Private layer turned off.

I made a Card scene with more than one Sprite2D for card images. The standard card back face is the lowest and its Visibility Layer is on for both Private and Public. The one for card front face starts off with only the Private layer on and Public layer off. It also has a function for altering its “publicness”, which just sets the Public layer on or off.

I had considered a design of using a secondary window for private information like hand and library and just not having those in the main window. I might still provide that as an alternative later on, but I would prefer to interact with just the main window, and for the most part is good to be more transparent with opponents about actions being taken (eg, being able to see if I am interacting with the content of the deck).

This is actually enough for using OBS itself to capture the second, Public-only window since it can be given a title for OBS to match for a Window Capture source, since it doesn’t matter if the second window is behind others - it is still captured as long as it has not been reduced to the taskbar. But… I am stubborn.

Sending to OBS Virtual Camera

The DLL I referenced before was based on C++ pieces from pyvirtualcam targeting OBS Virtual Cam and had a simple interface of some C-style exported functions added on for use by a different game engine since it didn’t have the means of using the C++ class within.

So I started off by just modifying these functions a bit and writing a C# wrapper for Godot to call.

The C++ DLL communicated via 2 shared memory buffers. One is for a sequence of frame “slots” so that the writer (C#) could be writing to one in parallel with the reader (virtual cam stuff) reading and sending another one out. The size of each frame in bytes was image_width * image_height * bytes_per_pixel, and then that multiplied by the number of frame slots - which in my case is 3. The other was a small flag buffer where each of the 3 bytes are used to communicate state of the corresponding frame, that way the reader and the writer did not step on each other’s toes. It’s also worth pointing out that a significant modification in the C++ code was the pixel format it was expecting, since its code for the other engine was expecting 4 bytes per pixel where one of them was for transparency, but in Godot when transparency of the Viewport is off, it renders a 3 bytes per pixel format of just RGB.

For the most part the memory work was not too hard since I could use a few tools from System.Runtime.InteropServices, although I did end up brushing up on my pointer knowledge. I could use Marshal.AllocHGlobal(frame_size * number_of_frames) to allocate an unmanaged buffer in memory, returning an IntPtr for the beginning of that buffer.

To address cleanup first, since I had to allocate the buffers before calling the DLL’s start_camera() function and pass the buffers to it, there are things it checks that may result in the camera not starting. Additionally, some other places like stopping the camera, and overriding the Dispose(bool) method for doing cleanup. Each of these places needed to do code similar to this for each buffer:

if (buffer != IntPtr.Zero) {
    Marshal.FreeHGlobal(buffer);
    buffer = IntPtr.Zero;
}

A _current_frame field was used for tracking and cycling through the frames and flags, and is used in math for calculating pointer offsets using IntPtr.Add() (it is, of course, zero-indexed). Here is more-or-less the content of the C# SendFrame function:

public bool SendFrame(Image frame) { //Godot.Image
    if (!get_is_camera_running())
        return false;

    byte[] frame_bytes = frame.GetData();

    if (frame_bytes == null)
        return false;

    if (frame_bytes.Length != _frame_size)
        return false;

    IntPtr cur_flagbuf = IntPtr.Add(_flagsbuffer, _current_frame);

    byte flag = Marshal.ReadByte(cur_flagbuf);

    if (flag != 0) // not ready-for-write
        return false;

    Marshal.WriteByte(cur_flagbuf, 1); // writing

    IntPtr cur_framebuf = IntPtr.Add(_buffer, _current_frame * _frame_size);
    Marshal.Copy(frame_bytes, 0, cur_framebuf , _frame_size);

    Marshal.WriteByte(cur_flagbuf, 2); // ready-for-read

    _current_frame = (_current_frame + 1) % number_of_frames;

    return true;
}

Finally (minus certain glossed-over, nitty-gritty details), a GDScript in the secondary window is doing the below (where GlobalVCH is the global instance of the helper object from the C#) and I have project settings set to cap frame rate because 60 is plenty for this kind of application.

func _process(delta: float) -> void:
    if GlobalVch.GetIsCameraRunning():
        await RenderingServer.frame_post_draw
        var img = get_viewport().get_texture().get_image()
        GlobalVCH.SendFrame(img)

Hate to bury this tidbit at the very end, but I found it odd that if I did img.get_data() in the GDScript I would get null and not a PackedByteArray, but first passing the Image to C# worked for frame.GetData(). :person_shrugging:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.