How to get the default light behavior with custom fragment shader that is setting the NORMAL for some pixels? I would like the defualt behavior to stay in those pixels that I do not set the NORMAL for.
This is the behavior I want to stay intact, the light is coming from top and the light and shadow work correctly in the ‘cave’ with darker color:
The light seems to touch the bottom of the cave on the expected areas, but the light is not visible on the cave background.
How the get the light visible on the background (i.e. not care about the normals) but also get the normals working for the edges?
Here is the shader code:
vec3 get_normal(vec4 sample) {
...some logic to calculate normal at the sampled point
return calculated_normal
}
void fragment() {
if (COLOR.g > 0.5) {
// Red channel to indicate foreground
vec2 uv = (UV + vec2(1.0 / float(grid_size))) * vec2(float(grid_size)) / vec2(float(grid_size + 1));
vec4 sample = texture(cell_grid_texture, uv);
COLOR = vec4(base_color.rgb, 1.0);
NORMAL = get_normal(sample); // Commenting this line makes the background work as expected
} else if (COLOR.r > 0.5) {
//Green channel to indicate background
COLOR = vec4(base_color.rgb * 0.2, 1.0);
}
}
I think the main question is that what does godot do on the background when using NORMALs in the shader? It seems to be disabling or overriding something?
And the other question is that is where could I find the default shader code? I Guess I could replicate it manually to get the light appearing even with the normals set?
Okay good point, the images are from a sideview 2d game I’m making. It can be hard to understand it from the static pictures. The blue above is sky, then the lighter brown is the ground and the darker is a hole in the ground. The dark brown being the background of the hole, or cave. In the above picture there is the even darker area in the cave, because the overhang creates a shadow downwards, the shadow also leaks to the ground, which is different issue.
The ground is formed of a mesh, formed from polygons generated using marching squares. There is also light occluders inside the ground polygons, which create the shadow in the cave in the first picture.
I’ve been trying to make it look more interesting and started looking into shaders. The challenge I faced is that I tried to set the NORMALs for the pixels covering the ground (to get those edge highlights when light hits them) and I wanted to keep the shadows casted on the cave wall.
But If I set the NORMAL in the shader, it looks like the shadow disappears from the wall. But it actually is that the light is not affecting the background mesh (darker brown) anymore, since I guess it gets a default NORMAL value (even though I don’t set it for the pixels covering the cave background) which is not in line with the light. I Could set the normal pointing upwards for those pixels to fix the issue, but the light can also come in a different angle than directly down.
And by the way I can’t create the same cave every time, the shape of the surface is random and I manually ‘dig’ a hole in it for testing. Just so you understand why the shapes don’t match
Is this a 3d mesh, but all vertices are coplanar? Or is is it something else? Can you render a wireframe in a 3/4 view
Also, render NORMAL as ALBEDO in both cases to get the ideas what normal values actually are. Have in mind that NORMAL in fragment function is in camera space, while in vertex function it’s in object space.
It seems as soon as NORMAL is used inside the shader function, it is used for every fragment.
// part of void main{} in canvas.glsl
if (normal_used) {
vec3 light_vec = normalize(mix(vec3(direction, 0.0), vec3(0, 0, 1), light_array[light_base].height));
light_color.rgb = light_normal_compute(light_vec, normal, base_color.rgb, light_color.rgb, specular_shininess, specular_shininess_used);
} else {
light_color.rgb *= base_color.rgb;
vec3 light_normal_compute(vec3 light_vec, vec3 normal, vec3 base_color, vec3 light_color, vec4 specular_shininess, bool specular_shininess_used) {
float cNdotL = max(0.0, dot(normal, light_vec));
// cNdotL is then used as a multiplicator for calculating the return value
Since there’s no normal map, fragments without a set NORMAL are probably vec3(0.0) and calculating dot(normal, light_vec) returns 0.0, preventing any lighting.
Not sure if there’s a better way to solve this, but I would just set the background NORMALs to point upwards for now. If you want to change the light’s direction you could pass it into the shader and change the code to use it’s negative as NORMAL instead.
P.S.:
Or you could manually calculate LIGHT in the light shader (with cNdotL for foreground, without it for background), passing a bool from fragment{} to light{} to differentiate between them.
