How are normals & tangents stored when using RenderingServer.mesh_surface_update_vertex_region?

Godot Version

4.6

Question

From using mesh_surface_get_format_normal_tangent_stride and mesh_surface_get_format_offset I can tell that normals & tangents are located after the vertex data, and they use 4 bytes of data each. From looking at the engine source I think they’re using octahedral encoding with half-precision floats, but even when using duplicates of the in-engine octahedron_encode & octahedon_tangent_encode, my manually-written normals don’t behave the way the automatically-computed ones do. Is there something else happening here? Is there any documentation covering how to write normals & tangents in this fashion I’ve missed?

(Specifically, the relevant parts of my code are this:

func norm_to_oct (n : Vector3) -> Vector2:
	var o = Vector2(0,0)
	if n.z > 0.0:
		o.x = n.x
		o.y = n.y
	else:
		o.x = (1.0 - abs(n.y)) * (1.0 if n.x >= 0.0 else -1.0)
		o.y = (1.0 - abs(n.x)) * (1.0 if n.x >= 0.0 else -1.0)
	o.x = o.x * 0.5 + 0.5
	o.y = o.y * 0.5 + 0.5
	return o

func tangent_to_oct (v : Vector3, sign : float) -> Vector2:
	var bias = 1.0 / 32767.0
	var res = norm_to_oct (v)
	res.y = max(res.y, bias)
	res.y = res.y * 0.5 + 0.5;
	res.y = res.y if sign >= 0.0 else 1.0 - res.y
	return res

and then in the buffer-writing function:


	var normal = norm_to_oct(normals[ix])
	var tangent = tangent_to_oct(tangents[ix], 1.0)
	normaltangents.encode_half(ix * nt_stride + 0, normal.x)
	normaltangents.encode_half(ix * nt_stride + 2, normal.y)
	normaltangents.encode_half(ix * nt_stride + 4, tangent.x)
	normaltangents.encode_half(ix * nt_stride + 6, tangent.y)

where normaltangents is the PackedByteArray argument to mesh_surface_update_vertex_region)

This is way out of my skill level, but I think I found some relevant documentation for you

Shaders expect the numbers encoded into 16 bit unsigned integers, not into half precision floating points.

You don’t need to implement custom octahedron compression function. Godot already has this.

Use Vector3::octahedron_encode() to compress the normal/tangent into Vector2, multiply it by 65535 and then encode each component into two bytes using PackedByteArray::encode_u16()

That’s it! I had been using my own octahedral-encoding function because I saw the octahedron_tangent_encodefunction that also encodes the binormal/cross product direction, but I guess part of the engine processing of the .w tangent component involves just flipping the tangent (or something) if it would be negative? So it’s not necessary to actually put that part of the encoding into the vertex buffer; I can just octahedron-encode both of them the same way. It looks like I was overcomplicating things. Anyway, thanks for the help; it’s all working now.