Problems that arise when mixing matrix conventions

The articles written on this blog were created with the goal of teaching. Also, the information in the article may be incorrect, so please be careful.

There are cases where the matrix code is moved, but the object rotates in the opposite direction or the movement becomes strange. This usually occurs when Row Vector, Column Vector, storage order, and coordinate system conventions are mixed.

Row and column vectors

Row vector convention
v' = vM

Column vector convention
v' = Mv

Even if it is the same transformation, the matrix shape and combination order vary depending on which side the vector is multiplied on.

Row-Major is different from multiplication rule.

Row-Major and Column-Major refer to whether rows or columns are stored consecutively in memory. Using row vectors and using Row-Major storage do not mean the same thing.

procession
[ a b ]
[ c d ]

Row-Major Memory: a b c d
Column-Major Memory: a c b d

transpose matrix

Transpose swaps rows and columns.

[ a b c ]T   [ a d ]
[ d e f ]  = [ b e ]
              [ c f ]

If the expected layout of the CPU library and Shader are different, they may be transposed before upload. However, if you unconditionally insert Transpose without knowing the reason, it will be reversed again when combined with other code.

Also check the coordinate system

Left-Handed and Right-Handed coordinate systems affect the Z-axis direction, rotation direction, and projection matrix. DirectX function nameLH, RHrepresents this difference.

When encountering a matrix bug, it is better to write down the following first rather than just looking at the numbers.

Are vectors rows or columns?
In which direction is the multiplication?
Is memory storage Row-Major?
Is the coordinate system LH or RH?
Transpose in Shader?

Each of the rules is not difficult. The problem is mixing code from different protocols without explanation.