More accurate version of detecting a Node3D that is visible on the screen?

Help Programming
, ,
1

Godot Version

4.7.stable

Question

First of all, VisibleOnScreenNotifier3D is not the solution to this! For the thing I want to do, VisibleOnScreenNotifier3D is too inaccurate. I need a ‘pixel-perfect’ solution.

Raycasting also doesn’t work! Because, the raycast only goes to the ‘center’ of the node I’m raycasting to, which means that when, let’s say, the right side of a box is already on screen, the raycast still won’t detect, because the center of the node isn’t visible (I don’t know if I’m doing something wrong here).

I want to do the following in my project:

  1. I want to detect when a Node3D/StaticBody3D/MeshInstance3D/CollisionShape3D is visible on screen (whichever of the 3 nodes is easier to detect…)

  2. It needs to have like 99% accuracy, VisibleOnScreenNotifier3D is too inaccurate.

  3. It needs to work for shapes like spheres, not only a bounding box like the VisibleOnScreenNotifier3D has.

  4. The detection mustn’t work through walls, so when I’m behind a wall, the node obviously isn’t visible to the player’s eye, so the node won’t be detected.

My project: I want to recreate the effect of a sphere ball only rolling when it’s visible to the player’s eye. (with a RigidBody3D) When you look away from the ball, it stops rolling.

My current implementation is with VisibleOnScreenNotifier3D and occlusion culling, which works OK, but is super clunky at times, problems like when the camera clips through the wall, it will detect objects through walls even with occlusion culling enabled.

If anyone has a starting position for me to start programming this please give me some suggestions.

2

The first thing that comes to my mind would be to do it with the stencil buffer and a compositor effect. At a high level, this means that each visible pixel of the sphere would write to the stencil buffer, then a compute shader would check whether there are any at all (since this might have a little perfotmance impact, it might be better to only do this after checking the VisibleOnScreenNotifier3D), but probably there are better ways. Still, I’m not 100% sure you can access the stencil buffer in compute shaders. And of course, compute shaders themselves are complicated, expecially if it’s your first time using them

EDIT: And you would need to use mobile/forward + for rendering

EDIT 2: Even if there was no way to access the stencil buffer, you could still do a sphere intersection manually and compare the obtained depth to the screen depth texture. Needless to say, this would be even more complicated

3

I’d recommend considering asking for help with your current implementation before giving up on it. Because TBH, this reads to me like you can’t get VisibleOnScreenNotifier3D working.

Either way way, you haven’t given nearly enough detail about the problems you are facing for us to do more than throw ideas at you. So here’s one: Instead of using a RayCast3D, use a ShapeCast3D that’s a Cylinder projecting from the camera. Make it’s radius the same as the Sphere’s. It’ll immediately detect any part of the Sphere it can see. You just put the Sphere on its own physics layer, and set the ShapeCast3D’s physics mask to only detect that layer. Walls in the way will block it regardless of any visual occlusion culling being done by the camera.

Also, I recommend using a CharacterBody3D instead of a RigidBody3D, as that’s probably some of the “super clunky” issue you are having. You cannot stop and start a RigidBody3D in a single frame. It’s intended to have forces applied to it and let the physics engine handle things from there. Directly setting the velocity on a RigidBody3D causes weird things to happen, including causing it to jump around. If you want to control the velocity frame-by-frame, use a CharacterBody3D.

4

What is it you are trying to do? Can you give the “big picture” problem?

5

Another, way simpler, option would be to use multiple VisibleOnScreenNotifier3Ds to approximate the sphere as much as you need. I don’t know of any ideal sphere-filling box arrangement but you can probably find it online

6

Yes, the problem is that I’m a beginner, and I’ve never worked with compute shaders or compositer effects, I only know approximately how stencil buffers work. That’s why I’m asking for general starting positions…

I am using Forward+ for the project.

But anyway, I’ll maybe post a demo video here some time soon.

7

I’ll post a video soon here, trying it to show clearly. Please be patient.