Godot Version
4.4
Question
I need a sort of specific visual effect that a couple of different approaches can't quite create, due to seemingly missing feature support in a couple small ways. Here's what I need, and a couple different things I've tried:
I need a visual effect that projects an image onto the scene geometry (like a decal) and only on certain objects (like with a visual layer culling mask) and from a custom texture generated from a subviewport (which decals do not support).
So this is very nearly something a decal can do, but i NEED to be able to generate the texture from a subviewport and for some reason you cannot use a subviewport texture as a decal input texture, I do not know why.
To get around this problem, I completely reimplemented decals with a fragment shader. I wrote a custom shader that turns a mesh into a volume that projects a texture onto the geometry, and you can easily use a subviewport texture as a shader parameter. This works perfectly (better actually), except that shaders seemingly cannot detect render layers in the way that decals can (with the culling mask). Like what I would specifically need to do is be able to read from some kind of visual layer buffer so I know to discard fragments on objects from the wrong layer.
So we’re back to square one: native decals don’t work because of the subviewport incompatibility, and custom shaders support subviewport textures but can’t discriminate objects. I know that the next version of Godot has some kind of stencil buffer support but I really would like to avoid migrating my whole already graphically complicated project to a new version.
So that leaves me with either: resolve the subviewport problem somehow, or resolve the render layer problem somehow. Is there a way for me to covert a subviewport to a normal compatible texture at runtime (in a non expensive way)? Is there another way for me to discriminate objects that are behind my fragment in a shader without a stencil buffer? It really is annoying how restrictive decals are, it would be ideal if you could make a decal shader to actually custom define the way they are drawn.
I know that Godot can do what I need it to here because I have two partial solutions, any help is appreciated. Thanks