8-bit uint encoding is disintegrating somewhere in my shader

[bobthefairycat]

Godot Version

v4.2.1 stable

Question

I created some images that I want to use for masking my sprite. The images are encoded as RGBA images, with each channel having 8 bits of depth. I double checked the .png files I created, and they indeed have Bit depth as 32 in the windows properties viewer.

In the G channel of my images, I am setting a single bit to high and all other bits low to indicate a mask layer (let’s assume for now that only one mask layer is ever active, so there is only one high bit present). For example: 0b001 sets mask layer 1 (head) to active, and 0b010 sets mask layer 2 (shirt) to active. Although these values seem to be loading into the active node correctly, the shader is reading the wrong values from the image at the same coordinates. I am having trouble debugging where these bit values are being mangled in between the node I load the texture, and the shader where I try to read it. Relevant code snippets are below. I check the pixel at coordinates (100, 50) where I know the RGBA value (0-255) is (0, 1, 0, 255).

My understanding is that loading in the texture as RGBA8, then reading it in the shader user usampler2D should preserve the bit encoding I have used; texelFetch at the exact pixel coordinate should then give me the 8-bit, unsigned int encoding that I want.

I have also checked the mask_bin in the shader code below against 2u, 3u, and 4u. I wish I had an easy way to figure out what value is being read into mask_bin, because I know it’s not 0u (see comment in the code), but I don’t know what it is… any suggestions on how to debug, or where I might have misused typing would be appareciated.

extends CharacterBody2D

# _sprite is an AnimatedSprite2D, 
# set to inherit material from parent 
@onready var _sprite = $Anim 

var walk_frame_textures = Array()
var shader_material

func _ready():
    shader_material = material as ShaderMaterial
    
    # load each frame of the sprite as an RGBA 8-bit per channel image
    for i in _sprite.sprite_frames.get_frame_count(_sprite.animation):
        var curr_frame_texture = _sprite.sprite_frames.get_frame_texture(
            _sprite.animation, _sprite.frame
        )

        var texture_as_rgb8 = curr_frame_texture.get_image()
        texture_as_rgb8.convert(Image.Format.FORMAT_RGBA8)
        walk_frame_textures.push_back(texture_as_rgb8)

    var pixel_val = walk_frame_textures[0].get_pixel(100, 50)
    print(pixel_val.g8)  # prints 1 in output log

func _process(delta):
    shader_material.set_shader_parameter(
        "texture_sampler", walk_frame_textures[_sprite.frame]
    )
		

shader_type canvas_item;

uniform usampler2D texture_sampler : filter_nearest;

const uint lsb_only = uint(1);  // 0b01

void fragment() {
    uvec4 texture_rgba = texelFetch(texture_sampler, ivec2(100, 50), 0);
    uint mask_bin = (texture_rgba.g);
   
    /** checking against the mask being non-zero activates the condition, and
    the entire sprite turns red **/
    // if (mask_bin != 0u){
    if (mask_bin == (lsb_only)){ 
        COLOR.rgb = vec3(1, 0, 0);
    } else {
        COLOR.rgb = vec3(1, 1, 1);
    }

    /** the below line does in fact color my entire sprite red, 
    so the shader is confirmed to be applied to the node **/
    // COLOR.rgb = vec3(1, 0, 0); 
}

I am also providing the image here, but I am not sure if the web upload will do some kind of compression or the like to mess up the encoding.

[bobthefairycat]

Well, after revisiting this issue on github: Integer samplers in shaders don't seem to provide integers representing the integer values set in the texture · Issue #57841 · godotengine/godot · GitHub

It seems the thread describes exactly what is going on. The code example provided below colors the sprite green.

shader_type canvas_item;

uniform usampler2D texture_sampler : filter_nearest;

const uint lsb_only = uint(1);  // 0b01

void fragment() {
    uvec4 texture_rgba = texelFetch(texture_sampler, ivec2(100, 50), 0);
    uint mask_bin = (texture_rgba.g);

    float eps = 0.01;
    float mask_as_float = uintBitsToFloat(mask_bin);
    if (mask_as_float <= (1.0/255.0 + eps) && mask_as_float >= (1.0/255.0 - eps)){
        COLOR.rgb = vec3(0, 1, 0);
    }
}

Essentially, the shader is reading in the image texture that is loaded in the node as a uint8 representation of a number, scales it by 255, then converts that result to a floating point representation for values between 0 and 1. This (32-bit?) floating point encoding of a value between 0 and 1 is what is offered by the texture_sampler to texelFetch, just stored in a 32 bit width uint (not 8-bit like I initially thought – I assumed the GLSL standard for uints was the same as an unsigned char in C, but I was wrong…).

This weird encoding is incredibly annoying…

[bobthefairycat]

Leaving a note in case anyone stumbles across this and finds it helpful, since my original code also had bugs unrelated to the bit representation of the values.

Thanks to TV4Fun on GitHub, there is a clean workaround that seems to work without resorting to floating point comparisons: Integer samplers in shaders don't seem to provide integers representing the integer values set in the texture · Issue #57841 · godotengine/godot · GitHub

The bugs in my original code are as follows:

1. Repeatedly setting the sampler2D uniform via code for the node resulted in a subsequent call to get_parameter() returning null, meaning I should not be repeatedly updating the texture parameter in _process. Obviously, the solution is the use a sampler2DArray instead.
2. In my _ready() function, I was always loading the texture for my sprite frame 0 since I was not updating the index in the assignment statement:

var curr_frame_texture = _sprite.sprite_frames.get_frame_texture(
	_sprite.animation, _sprite.frame
)

Combining the “fix” for the bit representation and the fixes for 1 and 2, the node code is

extends CharacterBody2D

# _sprite is an AnimatedSprite2D, 
# set to inherit material from parent 
@onready var _sprite = $Anim 

var walk_frame_textures = Array()
var shader_material

func _ready():
	shader_material = material as ShaderMaterial
	for i in _sprite.sprite_frames.get_frame_count(_sprite.animation):
		var curr_frame_texture = _sprite.sprite_frames.get_frame_texture(_sprite.animation, i)
		var texture_as_rgb8 = curr_frame_texture.get_image()
		texture_as_rgb8.convert(Image.Format.FORMAT_RGBA8)
		walk_frame_textures.push_back(texture_as_rgb8)

	texture_2d_array.create_from_images(walk_frame_textures)

	shader_material.set_shader_parameter(
		"texture_sampler", texture_2d_array
	)

func _process(delta):
	shader_material.set_shader_parameter(
		"sprite_frame", _sprite.frame
	)

and the shader

shader_type canvas_item;

uniform sampler2DArray texture_sampler : filter_nearest;
uniform int sprite_frame;

const uint lsb_only = uint(1);

void fragment() {
	vec3 head_color = vec3(212, 200, 199) / 255.0;
	vec3 shirt_color = vec3(62, 70, 97) / 255.0;
	vec3 pants_color = vec3(150, 73, 66) / 255.0;

	ivec2 loc = ivec2(UV * vec2(textureSize(texture_sampler, 0).xy));
	vec4 texture_rgba = texelFetch(texture_sampler, ivec3(loc, sprite_frame), 0);
	uint mask_bin = uint(texture_rgba.g * 255.0);

	COLOR.rgb = vec3(
			float(mask_bin & lsb_only) * head_color +
			float((mask_bin >> 1u) & lsb_only) * shirt_color +
			float((mask_bin >> 2u) & lsb_only) * pants_color
	);