How to calculate Tangent and Binormal? - math

Question

Talking about bump mapping, specular highlight and these kind of things in OpenGL Shading Language (GLSL)
I have:
An array of vertices (e.g. {0.2,0.5,0.1, 0.2,0.4,0.5, ...})
An array of normals (e.g. {0.0,0.0,1.0, 0.0,1.0,0.0, ...})
The position of a point light in world space (e.g. {0.0,1.0,-5.0})
The position of the viewer in world space (e.g. {0.0,0.0,0.0}) (assume the viewer is in the center of the world)
Now, how can I calculate the Binormal and Tangent for each vertex? I mean, what is the formula to calculate the Binormals, what I have to use based on those informations? And about the tangent?
I'll construct the TBN Matrix anyway, so if you know a formula to construct the matrix directly based on those informations will be nice!
Oh, yeh, I have the texture coordinates too, if needed.
And as I'm talking about GLSL, would be nice a per-vertex solution, I mean, one which doesn't need to access more than one vertex information at a time.
---- Update -----
I found this solution:
vec3 tangent;
vec3 binormal;
vec3 c1 = cross(a_normal, vec3(0.0, 0.0, 1.0));
vec3 c2 = cross(a_normal, vec3(0.0, 1.0, 0.0));
if (length(c1)>length(c2))
{
tangent = c1;
}
else
{
tangent = c2;
}
tangent = normalize(tangent);
binormal = cross(v_nglNormal, tangent);
binormal = normalize(binormal);
But I don't know if it is 100% correct.

Answer 1

The relevant input data to your problem are the texture coordinates. Tangent and Binormal are vectors locally parallel to the object's surface. And in the case of normal mapping they're describing the local orientation of the normal texture.
So you have to calculate the direction (in the model's space) in which the texturing vectors point. Say you have a triangle ABC, with texture coordinates HKL. This gives us vectors:
D = B-A
E = C-A
F = K-H
G = L-H
Now we want to express D and E in terms of tangent space T, U, i.e.
D = F.s * T + F.t * U
E = G.s * T + G.t * U
This is a system of linear equations with 6 unknowns and 6 equations, it can be written as
| D.x D.y D.z | | F.s F.t | | T.x T.y T.z |
| | = | | | |
| E.x E.y E.z | | G.s G.t | | U.x U.y U.z |
Inverting the FG matrix yields
| T.x T.y T.z | 1 | G.t -F.t | | D.x D.y D.z |
| | = ----------------- | | | |
| U.x U.y U.z | F.s G.t - F.t G.s | -G.s F.s | | E.x E.y E.z |
Together with the vertex normal T and U form a local space basis, called the tangent space, described by the matrix
| T.x U.x N.x |
| T.y U.y N.y |
| T.z U.z N.z |
Transforming from tangent space into object space. To do lighting calculations one needs the inverse of this. With a little bit of exercise one finds:
T' = T - (N·T) N
U' = U - (N·U) N - (T'·U) T'
Normalizing the vectors T' and U', calling them tangent and binormal we obtain the matrix transforming from object into tangent space, where we do the lighting:
| T'.x T'.y T'.z |
| U'.x U'.y U'.z |
| N.x N.y N.z |
We store T' and U' them together with the vertex normal as a part of the model's geometry (as vertex attributes), so that we can use them in the shader for lighting calculations. I repeat: You don't determine tangent and binormal in the shader, you precompute them and store them as part of the model's geometry (just like normals).
(The notation between the vertical bars above are all matrices, never determinants, which normally use vertical bars instead of brackets in their notation.)

Answer 2

Generally, you have 2 ways of generating the TBN matrix: off-line and on-line.
On-line = right in the fragment shader using derivative instructions. Those derivations give you a flat TBN basis for each point of a polygon. In order to get a smooth one we have to re-orthogonalize it based on a given (smooth) vertex normal. This procedure is even more heavy on GPU than initial TBN extraction.
// compute derivations of the world position
vec3 p_dx = dFdx(pw_i);
vec3 p_dy = dFdy(pw_i);
// compute derivations of the texture coordinate
vec2 tc_dx = dFdx(tc_i);
vec2 tc_dy = dFdy(tc_i);
// compute initial tangent and bi-tangent
vec3 t = normalize( tc_dy.y * p_dx - tc_dx.y * p_dy );
vec3 b = normalize( tc_dy.x * p_dx - tc_dx.x * p_dy ); // sign inversion
// get new tangent from a given mesh normal
vec3 n = normalize(n_obj_i);
vec3 x = cross(n, t);
t = cross(x, n);
t = normalize(t);
// get updated bi-tangent
x = cross(b, n);
b = cross(n, x);
b = normalize(b);
mat3 tbn = mat3(t, b, n);
Off-line = prepare tangent as a vertex attribute. This is more difficult to get because it will not just add another vertex attrib but also will require to re-compose all other attributes. Moreover, it will not 100% give you a better performance as you'll get an additional cost of storing/passing/animating(!) vector3 vertex attribute.
The math is described in many places (google it), including the #datenwolf post.
The problem here is that 2 vertices may have the same normal and texture coordinate but different tangents. That means you can not just add a vertex attribute to a vertex, you'll need to split the vertex into 2 and specify different tangents for the clones.
The best way to get unique tangent (and other attribs) per vertex is to do it as early as possible = in the exporter. There on the stage of sorting pure vertices by attributes you'll just need to add the tangent vector to the sorting key.
As a radical solution to the problem consider using quaternions. A single quaternion (vec4) can successfully represent tangential space of a pre-defined handiness. It's easy to keep orthonormal (including passing to the fragment shader), store and extract normal if needed. More info on the KRI wiki.

Answer 3

Based on the answer from kvark, I would like to add more thoughts.
If you are in need of an orthonormalized tangent space matrix you have to do some work any way.
Even if you add tangent and binormal attributes, they will be interpolated during the shader stages
and at the end they are neither normalized nor they are normal to each another.
Let's assume that we have a normalized normalvector n, and we have the tangent t and the binormalb or we can calculate them from the derivations as follows:
// derivations of the fragment position
vec3 pos_dx = dFdx( fragPos );
vec3 pos_dy = dFdy( fragPos );
// derivations of the texture coordinate
vec2 texC_dx = dFdx( texCoord );
vec2 texC_dy = dFdy( texCoord );
// tangent vector and binormal vector
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
vec3 b = texC_dx.x * pos_dy - texC_dy.x * pos_dx;
Of course an orthonormalized tangent space matrix can be calcualted by using the cross product,
but this would only work for right-hand systems. If a matrix was mirrored (left-hand system) it will turn to a right hand system:
t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( n, t ); // orthonormalization of the binormal vector
// may invert the binormal vector
mat3 tbn = mat3( normalize(t), normalize(b), n );
In the code snippet above the binormal vector is reversed if the tangent space is a left-handed system.
To avoid this, the hard way must be gone:
t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( b, cross( b, n ) ); // orthonormalization of the binormal vectors to the normal vector
b = cross( cross( t, b ), t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );
A common way to orthogonalize any matrix is the Gram–Schmidt process:
t = t - n * dot( t, n ); // orthonormalization ot the tangent vectors
b = b - n * dot( b, n ); // orthonormalization of the binormal vectors to the normal vector
b = b - t * dot( b, t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );
Another possibility is to use the determinant of the 2*2 matrix, which results from the derivations of the texture coordinates texC_dx, texC_dy, to take the direction of the binormal vector into account. The idea is that the determinant of a orthogonal matrix is 1 and the determined one of a orthogonal mirror matrix -1.
The determinant can eihter be calcualted by the GLSL function determinant( mat2( texC_dx, texC_dy )
or it can be calcualated by it formula texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y.
For the calculation of the orthonormalized tangent space matrix, the binormal vector is no longer required and the calculation of the unit vector
(normalize) of the binormal vector can be evaded.
float texDet = texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y;
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
t = normalize( t - n * dot( t, n ) );
vec3 b = cross( n, t ); // b is normlized because n and t are orthonormalized unit vectors
mat3 tbn = mat3( t, sign( texDet ) * b, n ); // take in account the direction of the binormal vector

Answer 4

There is a variety of ways to calculate tangents, and if the normal map baker doesn't do it the same way as the renderer you'll get subtle artifacts. Many bakers use the MikkTSpace algorithm, which isn't the same as the fragment derivatives trick.
Fortunately, if you have an indexed mesh from a program that uses MikkTSpace (and no texture coordinate triangles with opposite orientations share an index) the hard part of the algorithm is mostly done for you, and you can reconstruct the tangents like this:
#include <cmath>
#include "glm/geometric.hpp"
#include "glm/vec2.hpp"
#include "glm/vec3.hpp"
#include "glm/vec4.hpp"
using glm::vec2;
using glm::vec3;
using glm::vec4;
void makeTangents(uint32_t nIndices, uint16_t* indices,
const vec3 *positions, const vec3 *normals,
const vec2 *texCoords, vec4 *tangents) {
uint32_t inconsistentUvs = 0;
for (uint32_t l = 0; l < nIndices; ++l) tangents[indices[l]] = vec4(0);
for (uint32_t l = 0; l < nIndices; ++l) {
uint32_t i = indices[l];
uint32_t j = indices[(l + 1) % 3 + l / 3 * 3];
uint32_t k = indices[(l + 2) % 3 + l / 3 * 3];
vec3 n = normals[i];
vec3 v1 = positions[j] - positions[i], v2 = positions[k] - positions[i];
vec2 t1 = texCoords[j] - texCoords[i], t2 = texCoords[k] - texCoords[i];
// Is the texture flipped?
float uv2xArea = t1.x * t2.y - t1.y * t2.x;
if (std::abs(uv2xArea) < 0x1p-20)
continue; // Smaller than 1/2 pixel at 1024x1024
float flip = uv2xArea > 0 ? 1 : -1;
// 'flip' or '-flip'; depends on the handedness of the space.
if (tangents[i].w != 0 && tangents[i].w != -flip) ++inconsistentUvs;
tangents[i].w = -flip;
// Project triangle onto tangent plane
v1 -= n * dot(v1, n);
v2 -= n * dot(v2, n);
// Tangent is object space direction of texture coordinates
vec3 s = normalize((t2.y * v1 - t1.y * v2)*flip);
// Use angle between projected v1 and v2 as weight
float angle = std::acos(dot(v1, v2) / (length(v1) * length(v2)));
tangents[i] += vec4(s * angle, 0);
}
for (uint32_t l = 0; l < nIndices; ++l) {
vec4& t = tangents[indices[l]];
t = vec4(normalize(vec3(t.x, t.y, t.z)), t.w);
}
// std::cerr << inconsistentUvs << " inconsistent UVs\n";
}
In the vertex shader, they are rotated into world space:
fragNormal = (model.model * vec4(inNormal, 0)).xyz;
fragTangent = vec4((model.model * vec4(inTangent.xyz, 0)).xyz, inTangent.w);
Then the binormal and world space normal are calculated like this (see http://mikktspace.com/):
vec3 binormal = fragTangent.w * cross(fragNormal, fragTangent.xyz);
vec3 worldNormal = normalize(normal.x * fragTangent.xyz +
normal.y * binormal +
normal.z * fragNormal);
(The binormal is usually calculated per pixel, but some bakers give you the option to calculate it per vertex and interpolate it. This page has information about specific programs.)