Understanding the Basics of Matrix (and Matrices)

Matrix (and Matrices) are one of the most fundamental tools in mathematics, widely used in fields such as AI, physics, and economics. Let’s break down what a matrix is, how it’s structured, and some of the key terms you’ll often encounter.

What Is a Matrix?

An $m \times n$ matrix with entries from a field $F$ is simply a rectangular array of numbers (or elements) arranged in rows and columns. It looks like this:

$$
\begin{pmatrix}
a_{11} & a_{12} & \cdots & a_{1n} \\
a_{21} & a_{22} & \cdots & a_{2n} \\
\vdots & \vdots & \ddots & \vdots \\
a_{m1} & a_{m2} & \cdots & a_{mn}
\end{pmatrix}
$$

Here:

Each $a_{ij}$ is an entry of the matrix (with $1 \leq i \leq m, 1 \leq j \leq n$).
The $i$-th row is the sequence $(a_{i1}, a_{i2}, \dots, a_{in})$.
The $j$-th column is the sequence $(a_{1j}, a_{2j}, \dots, a_{mj})$.

The entries where the row and column numbers match ($i = j$) are called the diagonal entries.

For example, consider the following $3 \times 3$ square matrix:

$$
A = \begin{pmatrix}
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9
\end{pmatrix}
$$

$a_{11} = 1$ (row 1, column 1)
$a_{22} = 5$ (row 2, column 2)
$a_{33} = 9$ (row 3, column 3)

So, the diagonal entries are 1, 5, 9. These run from the top-left to the bottom-right of the matrix.

Special Types of Matrices

Zero Matrix

If every entry in the matrix is zero, it is called the zero matrix, denoted by $O$.

Square Matrix

If the number of rows equals the number of columns ($m = n$), the matrix is called square.

The Kronecker Delta and Identity Matrices

A useful notation in linear algebra is the Kronecker delta, denoted by $\delta_{ij}$. It acts as a binary indicator based on whether two indices are equal:

$$
\delta_{ij} = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases}
$$

Defining the Identity Matrix

Using this notation, we can formally define an $n \times n$ identity matrix ($I_n$). An identity matrix is a square matrix where the entry in the $i$-th row and $j$-th column is given by the Kronecker delta:

$$
(I_n)_{ij} = \delta_{ij}
$$

Examples

This definition results in a matrix where the main diagonal consists of ones, while all other entries are zero:

$1 \times 1$ Identity: $I_1 = (1)$
$2 \times 2$ Identity: $I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$
$3 \times 3$ Identity: $I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$

These types of matrices play an important role in advanced linear algebra, including determinants, eigenvalues, and linear transformations.

Notation and Equality

In mathematical writing, matrices are usually denoted by capital italic letters such as $A$, $B$, or $C$. The entry in row $i$, column $j$ of matrix $A$ is written as $A_{ij}$.

Two matrices $A$ and $B$ of the same size ($m \times n$) are said to be equal if every corresponding entry matches, i.e.,

$$
A_{ij} = B_{ij} \quad \text{for all } 1 \leq i \leq m, \; 1 \leq j \leq n.
$$

Example:

$$
B = \begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix},
\quad
C = \begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix}
$$

Here, $B = C$ because each corresponding entry is the same:

Top-left: $1 = 1$
Top-right: $2 = 2$
Bottom-left: $3 = 3$
Bottom-right: $4 = 4$

But if even one entry is different, the matrices are not equal.

Example:

$$
D = \begin{pmatrix}
1 & 2 \\
3 & 5
\end{pmatrix}
$$

Matrix $D$ is not equal to $B$, because the bottom-right entries differ ($4 \neq 5$).

Matrices as a Vector Space

When we study linear algebra, one powerful idea is that matrices themselves can be seen as vectors in a vector space. The set of all $m \times n$ matrices with entries from a field $F$ (such as the real numbers $\mathbb{R}$) is denoted

$$
M_{m \times n}(F).
$$

This set becomes a vector space when we define two operations:

Matrix Addition: For matrices $A, B \in M_{m \times n}(F)$,

$$
(A + B){ij} = A{ij} + B_{ij}
$$

That is, we add them entry by entry.

Scalar Multiplication: For a scalar $c \in F$ and $A \in M_{m \times n}(F)$,

$$
(cA){ij} = c \cdot A{ij}
$$

That is, we multiply every entry of the matrix by the scalar.

With these operations, the set of all $m \times n$ matrices satisfies the vector space axioms, which means it is a vector space.

The Vector Space Conditions

For any matrices $A, B, C \in M_{m \times n}(F)$ and scalars $c, d \in F$:

Closure under addition: $A + B \in M_{m \times n}(F)$.
Commutativity of addition: $A + B = B + A$.
Associativity of addition: $(A + B) + C = A + (B + C)$.
Additive identity: There exists a zero matrix $O$ such that $A + O = A$.
Additive inverse: For each $A$, there exists $-A$ such that $A + (-A) = O$.
Closure under scalar multiplication: $cA \in M_{m \times n}(F)$.
Distributivity (scalar over vector addition): $c(A + B) = cA + cB$.
Distributivity (field addition over vector): $(c + d)A = cA + dA$.
Associativity of scalar multiplication: $(cd)A = c(dA)$.
Multiplicative identity of scalar multiplication: $1 \cdot A = A$.

Together, these 10 conditions prove that matrices form a vector space.

Example with $2 \times 2$ Matrices

Let’s consider simple $2 \times 2$ matrices with real number entries.

$$
A = \begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix}, \quad
B = \begin{pmatrix}
5 & 6 \\
7 & 8
\end{pmatrix}, \quad
c = 2.
$$

Closure under addition: $$
A + B = \begin{pmatrix}
1+5 & 2+6 \\
3+7 & 4+8 \end{pmatrix} \begin{pmatrix}
6 & 8 \\
10 & 12
\end{pmatrix} \in M_{2 \times 2}(\mathbb{R}).
$$
Closure under scalar multiplication: $$
cA = 2 \cdot \begin{pmatrix}
1 & 2 \\
3 & 4 \end{pmatrix} \begin{pmatrix}
2 & 4 \\
6 & 8
\end{pmatrix} \in M_{2 \times 2}(\mathbb{R}).
$$
Additive identity: $$
A + O =
\begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix}
+
\begin{pmatrix}
0 & 0 \\
0 & 0 \end{pmatrix} \begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix} = A.
$$
Additive inverse: $$
A + (-A) =
\begin{pmatrix}
1 & 2 \\
3 & 4
\end{pmatrix}
+
\begin{pmatrix}
-1 & -2 \\
-3 & -4 \end{pmatrix} \begin{pmatrix}
0 & 0 \\
0 & 0
\end{pmatrix} = O.
$$

The rest of the properties can be checked similarly. Since all vector space axioms hold, we conclude that the set of matrices forms a vector space.

Definition: Matrix Product

Here is another important definition for the product of matrices. If we have a matrix $A$ of size $m \times n$ (m rows, n columns) and a matrix $B$ of size $n \times p$ (n rows, p columns), their product denoted as $AB$, is a new matrix of size $m \times p$.

The formula is defined as:

$$
(AB){ij} = \sum_{k=1}^{n} A_{ik} B_{kj}
$$

Key Rule: Matrix multiplication is only possible if the number of columns in the first matrix equals the number of rows in the second matrix.

Example 1: Square Matrices ($2 \times 2$)

Let’s multiply two $2 \times 2$ matrices.

$$
A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}
$$

To find the product $AB$:

Row 1 $\cdot$ Col 1: $(1 \times 5) + (2 \times 7) = 5 + 14 = \mathbf{19}$
Row 1 $\cdot$ Col 2: $(1 \times 6) + (2 \times 8) = 6 + 16 = \mathbf{22}$
Row 2 $\cdot$ Col 1: $(3 \times 5) + (4 \times 7) = 15 + 28 = \mathbf{43}$
Row 2 $\cdot$ Col 2: $(3 \times 6) + (4 \times 8) = 18 + 32 = \mathbf{50}$

Result:

$$
AB = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}
$$

Example 2: Rectangular Matrices ($2 \times 3$ and $3 \times 2$)

Here, $A$ has 3 columns and $B$ has 3 rows, so multiplication is possible. The result will be a $2 \times 2$ matrix.

$$
A = \begin{pmatrix} 1 & 4 & -2 \\ 3 & 5 & -6 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 2 \\ -1 & 0 \\ 0 & 3 \end{pmatrix}
$$

To find the product $AB$:

Row 1 $\cdot$ Col 1: $(1 \times 5) + (4 \times -1) + (-2 \times 0) = 5 – 4 + 0 = \mathbf{1}$
Row 1 $\cdot$ Col 2: $(1 \times 2) + (4 \times 0) + (-2 \times 3) = 2 + 0 – 6 = \mathbf{-4}$
Row 2 $\cdot$ Col 1: $(3 \times 5) + (5 \times -1) + (-6 \times 0) = 15 – 5 + 0 = \mathbf{10}$
Row 2 $\cdot$ Col 2: $(3 \times 2) + (5 \times 0) + (-6 \times 3) = 6 + 0 – 18 = \mathbf{-12}$

Result:

$$
AB = \begin{pmatrix} 1 & -4 \\ 10 & -12 \end{pmatrix}
$$

Would you like me to explain how to calculate the determinant of these matrices as well?