To start out with restrtsictions; I need to work in Mobile mode only for my specific project and not Compatability. The normal map I got seems to be in object space and I hope to convert it without having to rebake it.
In my Visual Shader, I tried using different vectors for the Tangent, Binormal and Normal inputs shown in the picture below
First problem was that it somehow takes the lighting in the front half of the model and reflects a copy onto the backside in respect to Model space. There is only one sun so this is not desired.
Second problem is that when I play it with an animation and skeleton, the lighting seems to be fixed to when it was in rest pose like a texture instead of real time lighting.
What’s strange is when moving the skeleton/object as a whole such as rotating it, the lighting reacts closer to expected with shadows coming up in the right places
Normal, tangent and binormal cannot be constant. You need to calculate binormal and tangent per vertex from the vertex normal. Then project the object space normal onto that basis.
I’d be surprised if you needed to change the model’s vertex normals at all. You don’t want to bake the normal map texture but your trying to change it through vertex data? How did you come to this conclusion? How is the normal map not working as you intend?
That’s why I had to create the 3 vectors in the first picture and plug them in to their repective sockets. However, when I start playing the animation the lighting stays fixed to the surface like it was a flat texture
They are part of the vertex data so they’ll typically be in objects space i.e. they’ll appear in the vertex exactly how they are stored in the vertex buffer. However tangents and binormals may not be present. You need to make sure your mesh data contains them.
I’m not sure if the original mesh had those two. Wouldn’t they be calculated by the UV map?
Also, maybe a workaround would be to reference the mesh’s respective bone transformations such as rotation and use the change to rotate the normals, tangents and binnormals. Though I’m not sure how to do that in the Visual Shader editor
Best to produce them in your modelling app and make sure they are exported. You can also generate them by enabling it in the import options although you’ll have a bit less control over their exact orientation.
The key thing is to construct a per-vertex basis out of those 3 vectors and transform the object space normal by it, all done in the vertex shader.
In Blender, I tried exporting the meshes as an FBX instead of GLTF and enabled export of tangents, but unfortunately it doesn’t seem to have changed anything in Godot. Is there a way to see the normals of the vertices in the viewport?
I didn’t make the model, I got it online. And it’s not just this one; I had to import hundreds of models that all use object space normal maps and I do not go through every single one of them. If I could make one Visual Shader that I could automatically apply to each model, it would be a lot easier down the road.
I’m not sure exactly how to make the specific TBN matrix needed for this case. Could you help me make one, or would you need to see my file?
Make 3 input nodes, set them to tangent, binormal and normal respectively and plug them into transform compose node.
That said, you may not need to do all this if you don’t assign to Normal Map output. You can instead simply convert the your object space normal map sample to camera space and assign it to Normal output in the fragment shader, completely overriding the vertex normals.
However, I think I figured out why there were mirrored reflections on my first pass. In Godot, whenever I imported a texture as a normal map, it would just get rid of the blue channel and leave just red+green. Since this is an object space normal map, this would cause the back half of the mesh to have the same normals in object space as the front half. Right now I’m trying to keep the texture as color first and figure out how to calculate it as a Normal. Right now it looks this:
Import as a regular texture, not as a normal map.
You’ll also need to decode the sample component value range from (0, 1) to (-1, 1).
So the whole things would require doing the following in the fragment shader:
sample the object space texture
decode to (-1, 1) range
convert from object space to camera space by multiplying with the normalized 3x3 basis of the modelview matrix (assuming there is no non-proportional scaling on the object)
