Defining Matrix Rank
In linear algebra, the rank of a matrix is defined as the maximum number of linearly independent row vectors in the matrix (the row rank) or the maximum number of linearly independent column vectors (the column rank). One of the most beautiful theorems in matrix theory states that for any matrix, the row rank is always equal to the column rank.
Because rank measures the number of independent vectors, it is naturally limited by the physical size of the matrix. Let's look at why rank can never exceed the number of rows or columns.
The Dimension Bottleneck
Suppose you have a matrix A of size m × n (m rows and n columns). Let's analyze the rank limits from both directions:
- Row Constraint: The matrix has m row vectors. You cannot have more linearly independent rows than the total number of rows you started with. Therefore:
rank(A) ≤ m. - Column Constraint: The matrix has n column vectors. You cannot have more linearly independent columns than the total number of columns you started with. Therefore:
rank(A) ≤ n.
Since both conditions must be true simultaneously, the rank is bounded by the smaller of the two dimensions:
rank(A) ≤ min(m, n)
Examples of Rank Boundedness
Consider a 3 × 5 matrix (3 rows, 5 columns). Even though there are 5 columns, the rank can be at most 3. This is because the column vectors exist in a 3-dimensional space (R3). You cannot have more than 3 linearly independent vectors in a 3-dimensional space.
Conversely, consider a 5 × 2 matrix. The rank can be at most 2. Even though the columns are vectors in R5, you only have 2 of them. Therefore, you cannot span a space of dimension higher than 2.
Rank in RREF
When you reduce a matrix to its Reduced Row Echelon Form (RREF), the rank is simply the number of non-zero rows (rows containing a pivot). Since you only have m total rows, you cannot have more than m pivots. And since each column can contain at most one pivot, you cannot have more than n pivots. This visualizes why the rank is bounded by min(m, n).