Are these functions column-major or row-major? - math

I'm comparing two different linear math libraries for 3D graphics using matrices. Here are two similar Translate functions from the two libraries:
static Matrix4<T> Translate(T x, T y, T z)
{
Matrix4 m;
m.x.x = 1; m.x.y = 0; m.x.z = 0; m.x.w = 0;
m.y.x = 0; m.y.y = 1; m.y.z = 0; m.y.w = 0;
m.z.x = 0; m.z.y = 0; m.z.z = 1; m.z.w = 0;
m.w.x = x; m.w.y = y; m.w.z = z; m.w.w = 1;
return m;
}
(c++ library from SO user prideout)
static inline void mat4x4_translate(mat4x4 T, float x, float y, float z)
{
mat4x4_identity(T);
T[3][0] = x;
T[3][1] = y;
T[3][2] = z;
}
(linmath c library from SO user datenwolf)
I'm new to this stuff but I know that the order of matrix multiplication depends a lot on whether you are using a column-major or row-major format.
To my eyes, these two are using the same format, in that in both the first index is treated as the row, the second index is the column. That is, in both the x y z are applied to the same first index. This would imply to me row-major, and thus matrix multiplication is left associative (for example, you'd typically do a rotate * translate in that order).
I have used the first example many times in a left associative context and it has been working as expected. While I have not used the second, the author says it is right-associative, yet I'm having trouble seeing the difference between the formats of the two.

To my eyes, these two are using the same format, in that in both the first index is treated as the row, the second index is the column.
The looks may be deceiving, but in fact the first index in linmath.h is the column. C and C++ specify that in a multidimensional array defined like this
sometype a[n][m];
there are n times m elements of sometype in succession. If it is row or column major order solely depends on how you interpret the indices. Now OpenGL defines 4×4 matrices to be indexed in the following linear scheme
0 4 8 c
1 5 9 d
2 6 a e
3 7 b f
If you apply the rules of C++ multidimensional arrays you'd add the following column row designation
----> n
| 0 4 8 c
| 1 5 9 d
V 2 6 a e
m 3 7 b f
Which remaps the linear indices into 2-tuples of
0 -> 0,0
1 -> 0,1
2 -> 0,2
3 -> 0,3
4 -> 1,0
5 -> 1,1
6 -> 1,2
7 -> 1,3
8 -> 2,0
9 -> 2,1
a -> 2,2
b -> 2,3
c -> 3,0
d -> 3,1
e -> 3,2
f -> 3,3
Okay, OpenGL and some math libraries use column major ordering, fine. But why do it this way and break with the usual mathematical convention that in Mi,j the index i designates the row and j the column? Because it is make things look nicer. You see, matrix is just a bunch of vectors. Vectors that can and usually do form a coordinate base system.
Have a look at this picture:
The axes X, Y and Z are essentially vectors. They are defined as
X = (1,0,0)
Y = (0,1,0)
Z = (0,0,1)
Moment, does't that up there look like a identity matrix? Indeed it does and in fact it is!
However written as it is the matrix has been formed by stacking row vectors. And the rules for matrix multiplication essentially tell, that a matrix formed by row vectors, transforms row vectors into row vectors by left associative multiplication. Column major matrices transform column vectors into column vectors by right associative multiplication.
Now this is not really a problem, because left associative can do the same stuff as right associative can, you just have to swap rows for columns (i.e. transpose) everything and reverse the order of operands. However left<>right row<>column are just notational conventions in which we write things.
And the typical mathematical notation is (for example)
v_clip = P · V · M · v_local
This notation makes it intuitively visible what's going on. Furthermore in programming the key character = usually designates assignment from right to left. Some programming languages are more mathematically influenced, like Pascal or Delphi and write it :=. Anyway with row major ordering we'd have to write it
v_clip = v_local · M · V · P
and to the majority of mathematical folks this looks unnatural. Because, technically M, V and P are in fact linear operators (yes they're also matrices and linear transforms) and operators always go between the equality / assignment and the variable.
So that's why we use column major format: It looks nicer. Technically it could be done using row major format as well. And what does this have to do with the memory layout of matrices? Well, When you want to use a column major order notation, then you want direct access to the base vectors of the transformation matrices, without having them to extract them element by element. With storing numbers in a column major format, all it takes to access a certain base vector of a matrix is a simple offset in linear memory.
I can't speak for the code example of the other library, but I'd strongly assume, that it treats first index as the slower incrementing index as well, which makes it work in column major if subjected to the notations of OpenGL. Remember: column major & right associativity == row major & left associativity.

The fragments posted are not enough to answer the question. They could be row-major matrices stored in row order, or column-major matrices stored in column order.
It may be more obvious if you look at how a vector is treated when multiplied with an appropriate matrix. In a row-major system, you would expect the vector to be treated as a single row matrix, whereas in a column-major system it would similarly be a single column matrix. That then dictates how a vector and a matrix may be multiplied. You can only multiply a vector with a matrix as either a single column on the right, or a single row on the left.
The GL convention is column-major, so a vector is multiplied to the right.
D3D is row-major, so vectors are rows and are multiplied to the left.
This needs to be taken into account when concatenating transforms, so that they are applied in the correct order.
i.e:
GL:
V' = CAMERA * WORLD * LOCAL * V
D3D:
V' = V * LOCAL * WORLD * CAMERA
However they choose to store their matrices such that the in-memory representations are actually the same (until we get into shaders and some representations need to be transposed...)

Related

Julia: Turn Vector into multiple m x n matrices without a loop

Let's say I have a vector V, and I want to either turn this vector into multiple m x n matrices, or get multiple m x n matrices from this Vector V.
For the most basic example: Turn V = collect(1:75) into 3 5x5 matrices.
As far as I am aware this can be done by first using reshape reshape(V, 5, :) and then looping through it. Is there a better way in Julia without using a loop?
If possible, a solution that can easily change between row-major and column-major results is preferrable.
TL:DR
m, n, n_matrices = 4, 2, 5
V = collect(1:m*n*n_matrices)
V = reshape(V, m, n, :)
V = permutedims(V, [2,1,3])
display(V)
From my limited knowledge about Julia:
When doing V = collect(1:m*n), you initialize a contiguous array in memory. From V you wish to create a container of m by n matrices. You can achieve this by doing reshape(V, m, n, :), then you can access the first matrix with V[:,:,1]. The "container" in this case is just another array (thus you have a three dimensional array), which in this case we interpret as "an array of matrices" (but you could also interpret it as a box). You can then transpose every matrix in your array by swapping the first two dimensions like this: permutedims(V, [2,1,3]).
How this works
From what I understand; n-dimensional arrays in Julia are contiguous arrays in memory when you don't do any "skipping" (e.g. V[1:2:end]). For example the 2 x 4 matrix A:
1 3 5 7
2 4 6 8
is in memory just 1 2 3 4 5 6 7 8. You simply interpret the data in a specific way, where the first two numbers makes up the first column, then the second two numbers makes the next column so on so forth. The reshape function simply specifies how you want to interpret the data in memory. So if we did reshape(A, 4, 2) we basically interpret the numbers in memory as "the first four values makes the first column, the second four values makes the second column", and we would get:
1 5
2 6
3 7
4 8
We are basically doing the same thing here, but with an extra dimension.
From my observations it also seems to be that permutedims in this case reallocates memory. Also, feel free to correct me if I am wrong.
Old answer:
I don't know much about Julia, but in Python using NumPy I would have done something like this:
reshape(V, :, m, n)
EDIT: As #BatWannaBe states, the result is technically one array (but three dimensional). You can always interpret a three dimensional array as a container of 2D arrays, which from my understanding is what you ask for.

BLAS routine to compute diagonal elements only of a matrix product?

Say I have two matrices A and B. I want to compute the diagonal elements of the matrix product A * B and place them in a pre-allocated vector result.
Is there a BLAS (or similar) routine to do this as fast as possible?
There is no specific routine for that. However, you can use the following definition of matrix multiplication.
Consider C = AB, and aij, bij, cij to denote the (i,j)th element of the corresponding matrices. Without loss of generality, I will assume that all A,B,C are N x N dense matrices.
Then,
cij = sumk=0N-1 (aik, bkj)
Since you are interested only in the diagonal entries:
cii = sumk=0N-1 (aik, bki), for i=1,...,N
In other words, to calculate the ith diagonal matrix of matrix C you need to find a dot product between the ith row of matrix A and ith column of matrix B. That can be achieved by using a dot product BLAS level-1 function ?dot.
res = ?dot(n, x, incx, y, incy)
Let's assume that matrices A and B are stored column-wise and are accessible via pointers *A and *B (which hold N*N values), while *C is a preallocated storage for diagonal entries of matrix C (which holds N values).
The following loop should give you the diagonal:
for (int i=0;i<N;i++)
{
C[i] = ?dot(N,A[i],N,B[i*N],1);
}
Notice, that we are accessing the ith row of matrix A by passing the first element of the ith row: A[i], and using increment (incx) of N. In contrast, to access the ith column of matrix B we pass the first element of the ith column: B[i*N] and use increment of 1.
Notes:
if A,B, and C have different (but consistent with matrix multiplication) dimensions, only slight modifications will have to be applied.
if matrices are stored row-wise, the call to ?dot should be slightly changed
the pseudocode above uses a general ?dot function. In practice, it will be sdot or ddot for single- or double precision real numbers, and versions of ?dotu: cdotu and zdotu for complex numbers of single and double precision, respectively.
is it the most efficient, cache-friendly, etc-etc implementation? probably not, but it would surprise me if that becomes a bottleneck in an algorithm where NxN matrices A and B have been explicitly calculated anyway.

Generate Unique Combinations of Integers

I am looking for help with pseudo code (unless you are a user of Game Maker 8.0 by Mark Overmars and know the GML equivalent of what I need) for how to generate a list / array of unique combinations of a set of X number of integers which size is variable. It can be 1-5 or 1-1000.
For example:
IntegerList{1,2,3,4}
1,2
1,3
1,4
2,3
2,4
3,4
I feel like the math behind this is simple I just cant seem to wrap my head around it after checking multiple sources on how to do it in languages such as C++ and Java. Thanks everyone.
As there are not many details in the question, I assume:
Your input is a natural number n and the resulting array contains all natural numbers from 1 to n.
The expected output given by the combinations above, resembles a symmetric relation, i. e. in your case [1, 2] is considered the same as [2, 1].
Combinations [x, x] are excluded.
There are only combinations with 2 elements.
There is no List<> datatype or dynamic array, so the array length has to be known before creating the array.
The number of elements in your result is therefore the binomial coefficient m = n over 2 = n! / (2! * (n - 2)!) (which is 4! / (2! * (4 - 2)!) = 24 / 4 = 6 in your example) with ! being the factorial.
First, initializing the array with the first n natural numbers should be quite easy using the array element index. However, the index is a property of the array elements, so you don't need to initialize them in the first place.
You need 2 nested loops processing the array. The outer loop ranges i from 1 to n - 1, the inner loop ranges j from 2 to n. If your indexes start from 0 instead of 1, you have to take this into consideration for the loop limits. Now, you only need to fill your target array with the combinations [i, j]. To find the correct index in your target array, you should use a third counter variable, initialized with the first index and incremented at the end of the inner loop.
I agree, the math behind is not that hard and I think this explanation should suffice to develop the corresponding code yourself.

A more generalized expand.grid function?

expand.grid(a,b,c) produces all the combinations of the values in a,b, and c in a matrix - essentially filling the volume of a three-dimensional cube. What I want is a way of getting slices or lines out of that cube (or higher dimensional structure) centred on the cube.
So, given that a,b, c are all odd-length vectors (so they have a centre), and in this case let's say they are of length 5. My hypothetical slice.grid function:
slice.grid(a,b,c,dimension=1)
returns a matrix of the coordinates of points along the three central lines. Almost equivalent to:
rbind(expand.grid(a[3],b,c[3]),
expand.grid(a,b[3],c[3]),
expand.grid(a[3],b[3],c))
almost, because it has the centre point repeated three times. Furthermore:
slice.grid(a,b,c,dimension=2)
should return a matrix equivalent to:
rbind(expand.grid(a,b,c[3]), expand.grid(a,b[3],c), expand.grid(a[3],b,c))
which is the three intersecting axis-aligned planes (with repeated points in the matrix at the intersections).
And then:
slice.grid(a,b,c,dimension=3)
is the same as expand.grid(a,b,c).
This isn't so bad with three parameters, but ideally I'd like to do this with N parameters passed to the function expand.grid(a,b,c,d,e,f,dimension=4) - its unlikely I'd ever want dimension greater than 3 though.
It could be done by doing expand.grid and then extracting those points that are required, but I'm not sure how to build that criterion. And I always have the feeling that this function exists tucked in some package somewhere...
[Edit] Right, I think I have the criterion figured out now - its to do with how many times the central value appears in each row. If its less than or equal to your dimension+1...
But generating the full matrix gets big quickly. It'll do for now.
Assuming a, b and c each have length 3 (and if there are 4 variables then they each have length 4 and so on) try this. It works by using 1:3 in place of each of a, b and c and then counting how many 3's are in each row. If there are four variables then it uses 1:4 and counts how many 4's are in each row, etc. It uses this for the index to select out the appropriate rows from expand.grid(a, b, c) :
slice.expand <- function(..., dimension = 1) {
L <- lapply(list(...), seq_along)
n <- length(L)
ix <- rowSums(do.call(expand.grid, L) == n) >= (n-dimension)
expand.grid(...)[ix, ]
}
# test
a <- b <- c <- LETTERS[1:3]
slice.expand(a, b, c, dimension = 1)
slice.expand(a, b, c, dimension = 2)
slice.expand(a, b, c, dimension = 3)

What does matrix*vector mean in contrast to vector*matrix

if I do positionVector*worldMatrix the position is transformed into world space.
But what happens if I do it the other way around (worldMatrix*positionVector) in terms of 3d space?
I noticed the result is different to the first one. I already googled about matrix, math they explain a lot but not this one, at least I couldn't find it.
As others have indicated - swapping the order of the multiplication is equivalent to multiplying by the transpose. As it happens, rotation matrices are a special type of matrices known as orthogonal matrices this gets you a number of neat properties.
The most interesting is probably that the transpose of the matrix is its inverse. For your world transform multiplying by the inverse is equivalent to taking a position in world space and pulling it into the local coordinates of the object that transform is associated with.
As an example, consider a box oriented arbitrarily in the world - multiplying by the inverse world transform could (entirely application dependant of course :)) put you in a space where it is axis aligned, and if you were interested in looking for collisions with other objects doing the calculations in the box's local space would make this easier.
In matrixvector, your vector will be interpreted as a column vector. In vectormatrix, it will be interpreted as a row vector. 2x2 examples:
/ a b \ / e \ / ae+bf \
| | * | | = | |
\ c d / \ f / \ ce+df /
/ a b \
( e f ) * | | = ( ea+fc eb+fd )
\ c d /
As you can see, the result is different.
Incidentally, doing the one is the same as doing the other after transposing the matrix.
In terms of 3D space, if you consider one of the two options to be a linear transformation, I don't know if there is any sensible interpretation for the other one. This Wikipedia section says things about it, but it is beyond my understanding of linear algebra.
(matrix * vector) is equivalent to (vector * transpose(matrix))
Matrix math rules:
Given matrices A and B, with sizes MxN and OxP,
The matrix product A * B is only defined if N=O.
The result will be a matrix with size MxP.
Another important rule is that matrix multiplication is not commutative. A * B != B * A
Typically in computer graphics, the position vector is a 4x1 matrix, and the world view matrix is square, 4x4. Thus you should expect that pre-multiplying the world view matrix with the position vector would be undefined. The proper way to apply the world view matrix to the position vector is in the other order, pre-multiplying the position vector with the world view matrix. (I'm speaking mathematically, here)
For more fun with matrix math, check out this tutorial.

Resources