How to properly use affline inverse transform in shader?

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By ghsoares


I’m creating a custom water shader that I’m taking the distance from a vertex global position to a object position based in custom distance functions to primitive types, like a sphere.

This will be used to clip parts of the water that intersects hollow shapes, by any means, if some water is inside a ship, for example, I can set a shape, pass its parameters to the water shader (type, transform, extra parameters), so I can calculate the distance from a global vertex position to this shape and give the ALPHA to 0.0 if the distance is less or equal to zero, so it makes the illusion that no water plane is inside the ship.

The problem is that I’m taking in account the transform of the object, so it matches in the shader, but I don’t know what I’m doing wrong here.

Basically I’m transforming the vertex position into the object transform using an inverse transform, and then using the distance functions to properly calculate then, but the transformation is not doing it right with the shader I have now. Maybe there’s an obvious mistake that I can’t say, but this is why I’m asking here so someone can help me.

This is my function to calculate the distance from point to sphere with transformation:

float distance_to_sphere(vec3 pos, float radius, mat4 inv_transform) {
	//This is the part where I transform the "pos" to be able to match the
	//object's transform
	pos = (inv_transform * vec4(pos, 1.0)).xyz;
	return max(0.0, length(pos) - radius);

This is the input of the function (in fragment I’m getting the global position of the vertex):

vec3 pos = (CAMERA_MATRIX * vec4(VERTEX, 1.0)).xyz;
float box_dist = distance_to_sphere(pos, 4.0, transform);

For the “transform” argument, is a uniform that I manually insert into the editor, using the object’s “global_transform.affine_inverse()” transform that, in theory, transform a point in world position to local position, so I pass as uniform.

The object’s transform is:

enter image description here

The “transform” uniform (affline inverse) is:

enter image description here

Outputing the results to albedo, comparing to the object, it shows something like this:
enter image description here

As you can see, the “shadow” center is not centered to the object’s center, and I don’t know what I’m doing wrong. If I try the same thing, but in a shader adapted to 2D, it outputs right, the function code is:

float distance_to_circle(vec2 pos, float radius, mat3 transform) {
	vec2 transformed_pos = (transform * vec3(pos, 1.0)).xy;
	return max(0.0, length(transformed_pos) - radius);

And the output:
enter image description here

But in this case, the red area shows bigger because the radius is a bit higher to be able to see if is centered.

So, to resume: How in a Spatial Shader, I can inverse transform a global position, so I can use it to calculate primitive shapes distances?

:bust_in_silhouette: Reply From: tshrpl

why use that transform uniform? i think inverse(WORLD_MATRIX) does the trick

uniform vec3 HOLE_POSITION : hint_vector? = vec3(0.);
uniform float HOLE_RADiUS : hint_range(1,10)? = 2.;

float distance_to_sphere(vec3 p, vec3 o, float k) {
	return length(p-o) - k;

void fragment() {
	// vec3 VERTEX_WORLD = (CAMERA_MATRIX * vec4(VERTEX, 1.0)).xyz;
	// vec3 VERTEX_LOCAL = (inverse(WORLD_MATRIX) * vec4(VERTEX_WORLD, 1.0))).xyz;
	VERTEX = (inverse(WORLD_MATRIX) * CAMERA_MATRIX * vec4(VERTEX, 1.0))).xyz;
	float f = distance_to_sphere(VERTEX, HOLE_POSITION, HOLE_RADIUS);

	ALBEDO = vec3(f);