Can't pack 32-bit int into a Color channel, for custom vertex data

Help Shaders
, ,
1

Godot Version

4.6.1

Question

A bit of a technical one, but I’m trying to pack 32 bits worth of data into each 32 bit channel of a Color, and use that as Custom Data for a vertex shader.

I’m guessing this is a limitation of floating point values, but the Color format seems to lose the 4 smallest bits in each of its channels, when set manually. This means I can only use 28 bits worth of data, which complicates my use case for this a fair bit!

For context, I’m using the SurfaceTool for custom mesh generation, so I believe I only have access to the Color variable as a method of passing Custom Data into a given vertex. The shader in question decomposes the bits in each 32 bit Color, and uses sets of 4 or 8 bits from them for various uses.

Is there’s any known way to pass a full set of 32 bits into a Color, or otherwise into the Custom Data of a vertex? Unless I’m misunderstanding something, I can’t see a way to get any use out of the last 4 bits per channel of Custom Data otherwise, in any circumstance.

Here’s a sample script for demonstrating the problem. As you can see, the interger bitmask stores the full 32 bits of data, as does each channel of a Vector4i, but Color seems to drop the final four bits.

extends Node3D

func print_bits(value: int, padding: int = 32) -> String:
	return String.num_uint64(value, 2).pad_zeros(padding)
	
func _ready():
	# Bitmask containing 32 bits of data
	var base_bitmask = 0b0001_1010_0000_1100_1000_1011_1010_0110
	
	# A Color and Vector4i, to test formats for packing bitwise data into channels
	var base_colour = Color(0.0, 0.0, 0.0, 0.0)
	var base_vector_int = Vector4i(0, 0, 0, 0)
	
	# Sets the first data channel of the Color or Vector4 to store the bitmask
	base_colour.r = base_bitmask
	base_vector_int.x = base_bitmask
	
	# Display actual bitwise data stored in each format
	print ("BASE BITMASK:  ", print_bits(base_bitmask, 32))
	print ("\nCOLOR BITS R:  ", print_bits(base_colour.r, 32))
	print ("VECTOR BITS X: ", print_bits(base_vector_int.x, 32))

Output:
image

Note that the final 4 bits of the Color are simply lost. I’m unsure why this is, or how to work around it. Any help would be very welcome.

2

Did you try with Vector4 instead of Vector4i?
I may be wrong but maybe it’s the datatype and IEEE 754. May be completely unrelated, but it is worth a try

3

Yeah, any floating-point-based data type seems to have the same limitation on losing the least significant bits sometimes.

At a guess, I think it’s something simplifying the value automatically when set; sometimes the 4 least-significant bits become 0000, and sometimes they become 1000, based on whatever 4 bits are supposed to be there.

I assume floats just have some simplification or conversion that truncates those bits, even though Colors as Custom Values aren’t really meant to be storing a floating point.

4

You can’t pass bits reliably via the color attribute. Use a custom attribute with an integer data format.

The loss of bits happens because you’re converting a large number to a single precision floating point. GDScript floats are double precision (64 bits, C++ double) but Color components are internally stored in single precision (32 bits, C++ float).

So when you do:

var x := 0b0001_1010_0000_1100_1000_1011_1010_0110
print(x)
print(float(x))
var color := Color(x, 0, 0, 0)
print(color.r)

The output will be:

437029798
437029798.0
437029792.0

The first one is the original integer. The second one is converted to GDScript float, and the third one is converted to a standard 32 bit float. This may not happen with small numbers but if you’re packing bits the notion of big/small numbers is of no significance.

1 Like
5

Ah yeah, I was afraid the precision loss would be unavoidable. Too bad there’s no way I can see to set the bits of a floating-point directly, the way there is for an integer, but I suppose it would be pretty unconventional.

Since SurfaceTool only provides a method for writing Color-based data to a vertex’s custom data, looks like I’m going to have to rebuild my mesh generation system around editing ArrayMesh arrays directly.

I appreciate the guidance!

6

You wouldn’t be able to do it like that with ArrayMesh either since it takes PackedColorArray as an input which again converts to 32 bit floats. You do need to use ArrayMesh though with a custom attribute. There you have a choice of data formats so you can pack the bits directly.

7

Oh, I was planning to use the ARRAY_CUSTOM0 through 4 as PackedByteArrays, but it sounds like the ArrayCustomFormat types provided may also all be floating-point based? That’s really too bad, if so.

Assuming you meant something else, do you know of any resources to help me get started on creating custom attributes? The lower-level Mesh documentation is a little hard to get started with, and I’ve not found anything yet.

Either way, I might just sacrifice the last 8 bits of the Color floats, and just pack the shader data a bit less tightly, since it’s starting to look like this might involve a significant overhaul. Oof.

8

What exactly you need to send to the shader?

9

I’m working on a voxel-based 3D dual grid terrain system, that stores generation data in voxels, and passes the relevant data into mesh vertices. It’s primarily used for texturing and recoloring each corner to match the data grid, since each mesh can represent up to 8 different textures/colors/etc.

image
image1807×917 222 KB

image
image1816×930 208 KB

Right now, each vertex needs 8x 8-bit Sprite Ids, 8x 4-bit Variant Ids, and a 4-bit Overlay Sprite Id.

Bit packing multiple pieces of data per Color channel opens up more options than using 1 Color channel per piece of data. I had plans to pack in a few more features (palette swapping, etc.), but that depends how many bits of data end up available.

1 Like
10

Hm, I looked at custom formats and there seems to be no way to go around floats, at least I couldn’t find it. Godot will convert all attribute data to floats anyway and deliver them as vec4s into the shader.

1 Like
11

It’s no worries. I think I can scope down my ambitions a bit, and stick to packing in the safe 24 - 28 most significant bits per float. I do wish there was a way to write to a specific bit of a floating point value, but it feels too hacky to look into it as a core feature, haha.

I appreciate you looking into it, and for your help in all this!

1 Like
12

The closest I can think of is to pack your bits directly into a PackedByteArray, reinterpret them to PackedFloat32Array using PackedByteArray::to_float32_array(), then just try to send those floats as attributes and unpack them in the shader using unpackUnorm*() functions.

It could work in theory but I don’t know if to_float32_array() would just accept straightforward NaNs. The documentation is vague on this. It only says “If the original data can’t be converted to 32-bit floats, the resulting data is undefined”. Not sure if the bits are preserved in that case. They might not be.

The only other solution is what you’re currently doing. Encode in some way that tolerates precision limitations of 32 bit floats.

2 Likes
13

Oh, I was hoping that would work! It’s a clever idea.

For future ref: I did a quick test of converting a PackedByteArray without sensible floats in it, via to_float32_array(). It seems to just return bits that are all 0 when it’s undefined. Ah well.

1 Like
14

It appears to be working according to my test. So you can actually set float bits this way. Here’s a little test that tries out various 4-byte configurations that are guaranteed to be NaN:

for i in 10:
	var nan := 0b0111_1111_1000_0000_0000_0000_0000_0000
	nan |= randi_range(0, 0b1111_1111_1111_1111_1111)
	var b := PackedByteArray()
	b.resize(4)
	b.encode_u32(0, nan)
	var f := b.to_float32_array()
	print("\n")
	print(b)
	print(f)
	b.encode_float(0, f[0])
	print(b)

Output:

[151, 251, 140, 127]
[nan]
[151, 251, 204, 127]


[251, 174, 142, 127]
[nan]
[251, 174, 206, 127]


[255, 148, 129, 127]
[nan]
[255, 148, 193, 127]

etc...

The bits seem to be preserved.

3 Likes
15

You probably don’t want to involve the Color object into this because it may not like those NaNs and could attempt to zero them.
Best to just convert bytes to a packed float array and send that directly as a custom attribute using ARRAY_CUSTOM_RGBA_FLOAT data format.

Looks like it could work.

1 Like
16

Oh! I must’ve filled my PackedByteArray incorrectly, because that does seem to be preserving the bits after the conversion.

Even though I already finished converting the data over, this is a neater and more efficient approach. I think I’ll give it a shot. Thanks so much for this!

2 Likes