You should first work with zeroed out-height map, and adjust everything so that distances/scale in both shaders match.
If I scale map size without height polygons, this postprocess overlay does not change at all. Seems only distances between objects on heightmap change, and if I remove the objects (the polygons) output image remains the same no matter what plane size I choose. (Like, no matter how big is the plane mesh, I always project on same floor point, as it is aligned with Y = 0 3d plane)
You need to make your depthmap buffer floating point to store the actual correct values. Otherwise itâll just clamp stuff to 0-1 range and remap to 8 bits per channel ints. To do this enable 2D HDR for the viewport.
For debug preview I just altered the last line in your shader to:
COLOR.rgb = vec3(step(0.0001, distance_error));
Youâd still want to raymarch that depthmap for this to properly handle any arbitrary case.
Ok, thank you, I will try this and write back soon.
It works, thank you. I spent 3 days trying to debug this with my friend and we tested different encodings which all seem to produce little different results, though we discarded the encoding to be the main problem because it seems difference in tone we get from sampled and calculated value is much higher than encoding error. Could you please explain how clamping stuff to 0-1 range and remapping to 8 bits per channel ints and back could produce errors greater than 1/255 of float?
Like, if the error was equal to float to unsigned byte encoding-decoding error, I expect this difference to be equal ~1/255, which would not produce grey color we see, but almost black image, like one we have, after we enable HDR 2D. I disabled all postprocessing and tonemapping on the viewport so why could the error explode?
Also I got very good shadows just enabling HDR 2D, without any raymarching. The divergence now is literally zero. Thanks again @normalized. I didnât even know you can render to float format texture in Godot. I used glsl shader with RenderingServer for this.
Well it depends on what your flashlight_projection matrix does. I havenât looked at it. Itâs likely the range error not encoding error. It could also be automatic linear->srgb conversion for non hdr viewport buffers. Enabling hdr ensures that the exact floating point values your shader outputs are written into the texture. Standard shadow maps are also typically stored that way.
Yeah this may work as an approximation, especially if the exact elevation of ocluders is not really the factor in this top-down view (in which case you can simplify even further and have a single dimensional depth texture). Just be aware that you could encounter some weirdness if the elevation is important and there are many overlapping ocluders.