🧭 Inverse, Rank, and Span

By Better Wise

Where This Fits

In the last lesson you drew a matrix as a transformation of space, and matrix multiplication as doing one transformation after another. That raises two follow-up questions. Once space has been moved, can it always be moved back? And how much of space does the transformation actually reach? Inverse, rank, and span answer exactly those questions, and they are the vocabulary the rest of this module runs on.

By the end of this lesson you can:

  • describe the span of a matrix as the set of every point its transformation can reach;
  • compute rank in NumPy and read it as the number of independent directions that survive;
  • use the determinant as an area meter and as an invertibility test;
  • compute and verify an inverse in NumPy, and recognize when none exists.

Span: Everything a Transformation Can Reach

A matrix $A$ with $n$ columns takes an input vector $x$ in $\mathbb{R}^n$ and returns $Ax$. Feed it every possible input and collect every output. That collection is the span of the columns of $A$, also called the _____.

The picture is direct. Columns are output directions: weight 1 on column 1 and 0 on the others returns column 1 itself. Mixing weights returns mixtures. So the reachable set is precisely all combinations of the columns.

Two columns pointing in different directions sweep out a plane. Two columns on the same line sweep out only that line. This matrix is a genuine squasher:

$$ M = \begin{pmatrix} 1 & 2 \ 2 & 4 \end{pmatrix} $$

The second column $(2,4)$ is twice $(1,2)$, so every output is a multiple of $(1,2)$. Everything lands on the line $y = 2x$. A 2D plane collapses to a 1D line, and the matrix can never send a point off that line.

flowchart TD
    In["📥 all inputs x in R^2"] --> M["🔄 matrix M = [[1,2],[2,4]]"]
    M --> Out["➡️ outputs: only the line y = 2x"]
    Out --> Note["span = column space"]

Note: Span describes where outputs live, not where inputs come from. The inputs still fill the whole plane, even when the outputs do not.

Practice

  1. For $A = \begin{pmatrix} 1 & 1 \ 1 & 1 \end{pmatrix}$, what is the span of its columns?
  2. For $A = \begin{pmatrix} 1 & 0 & 0 \ 0 & 1 & 0 \end{pmatrix}$, what is the span of its columns?

Click to reveal the answers

  1. The line $y = x$, since both columns are $(1,1)$.
  2. All of $\mathbb{R}^2$: the two independent columns $(1,0)$ and $(0,1)$ reach every point. The third column adds nothing new.

Rank: How Many Directions Survive

Rank is the number of linearly independent columns, which equals the dimension of the span. It counts the distinct directions a transformation actually keeps.

  • rank 2 for a 2x2 matrix with non-parallel columns: the span is a plane;
  • rank 1: the span is a line;
  • rank 0: only the zero matrix, whose span is a single point.

For an $m \times n$ matrix, the rank can never exceed _____. A wide matrix (more columns than rows) is guaranteed to have dependent columns; a tall matrix cannot exceed its number of rows.

One mirror fact is worth keeping: the rank also tells you how much you lose. The quantity $n - \text{rank}$ is the dimension of the _____, the set of inputs that get sent to zero. For $M$ above, rank 1, so a whole line of inputs vanishes: check that $M(2,-1) = (0,0)$.

import numpy as np
M = np.array([[1., 2.], [2., 4.]])
np.linalg.matrix_rank(M)   # 1

matrix_rank uses a tolerance based on the SVD, so it tolerates floating-point noise instead of demanding exact zeros.

Practice

  1. What is the rank of np.array([[1., 2., 3.], [2., 4., 6.]])?
  2. A 3x3 matrix has rank 2. How large is its null space?

Click to reveal the answers

  1. Rank 1: row 2 is exactly twice row 1, so there is only one independent direction.
  2. Dimension 1, since $3 - 2 = 1$. The matrix squashes one direction to zero.

The Determinant as an Area Meter

The determinant is defined only for square matrices. Read it as the signed factor by which the transformation scales area (in 2D) or volume (in 3D). The sign records whether orientation flipped, as in a reflection.

For a 2x2 matrix, the determinant is _____:

$$ \det \begin{pmatrix} a & b \ c & d \end{pmatrix} = ad - bc $$

Watch the unit square: corners $(0,0)$, $(1,0)$, $(0,1)$, $(1,1)$. Under $A = \begin{pmatrix} 2 & 0 \ 0 & 3 \end{pmatrix}$ it becomes a 2 by 3 rectangle with area 6, which is exactly $\det A$. Scale by $\frac{1}{2}$ in both directions and the area factor drops to $\frac{1}{4}$.

The key consequence: a determinant of zero means area has been squashed to nothing, some dimension collapsed, the columns are dependent, and the rank is below $n$. So a square matrix is invertible exactly when its determinant is _____.

Practice

  1. Compute $\det \begin{pmatrix} 3 & 1 \ 2 & 4 \end{pmatrix}$. What does the answer say about areas?
  2. Compute the determinant of a 90 degree rotation $\begin{pmatrix} 0 & -1 \ 1 & 0 \end{pmatrix}$.

Click to reveal the answers

  1. $12 - 2 = 10$. Areas are multiplied by 10, and the matrix is invertible.
  2. $0 \cdot 0 - (-1)(1) = 1$. Rotations preserve area and orientation.

The Inverse: Running a Transformation Backwards

If $A$ sends $x$ to $Ax$, then $A^{-1}$ sends $Ax$ back to $x$. In symbols, $A^{-1}A = AA^{-1} = I$. Geometrically the inverse is the transformation that undoes $A$: rotate 90 degrees and the inverse rotates 90 degrees the other way; stretch by 2 and the inverse shrinks by $\frac{1}{2}$.

An inverse exists only when nothing has been squashed: a square matrix with nonzero determinant, that is, full rank. If the plane collapsed onto a line, many inputs share the same output, and no function can send that output back to all of them at once.

B = np.array([[2., 1.], [1., 3.]])
np.linalg.inv(B)                          # fine, det = 5
np.linalg.solve(B, np.array([1., 2.]))    # preferred way to solve Bx = b
# np.linalg.inv(M)                        # LinAlgError: Singular matrix

Practice

  1. Compute the inverse of $\begin{pmatrix} 2 & 0 \ 0 & 4 \end{pmatrix}$ by hand, then confirm with NumPy.
  2. Without running code, predict what np.linalg.inv(np.array([[1., 2.], [2., 4.]])) does and why.

Click to reveal the answers

  1. Scaling by 2 and 4 is undone by scaling by 0.5 and 0.25, so the inverse is $\begin{pmatrix} 0.5 & 0 \ 0 & 0.25 \end{pmatrix}$. Verify that B @ np.linalg.inv(B) gives the identity.
  2. It raises LinAlgError, because the determinant is 0 and the matrix is rank 1.
Question Test Answer for the squasher $M$
How much space is reached? rank 1, a line
Is area squashed away? determinant 0
Can we undo it? inverse no, singular

Rules of Thumb and Numerical Traps

  • Test invertibility with np.linalg.matrix_rank or np.linalg.det, not with a guess from how the entries look. Small entries do not imply singular; dependent columns do.
  • Never test det == 0 in floating point on large matrices. Determinants grow and shrink wildly with matrix size. Use rank, or check the condition number with np.cond.
  • A matrix can be technically invertible and numerically hopeless: det near $10^{-18}$ means the inverse amplifies noise enormously.
  • Prefer np.linalg.solve(B, b) over np.linalg.inv(B) @ b. It is faster and more stable, and it fails loudly for singular systems.
  • Non-square matrices have no inverse, but they always have a meaningful rank. Do not conflate the two ideas.
  • Sketch the columns before computing anything. If they point the same way, you already know the rank is 1.

What to Carry Forward

  • Span is everything the transformation can reach; it is built from the columns.
  • Rank is the dimension of that span, and $n - \text{rank}$ is the amount of space sent to zero.
  • The determinant measures area or volume scaling, and zero means a collapse and no inverse.
  • The inverse exists only for square, full-rank matrices, and it undoes the transformation.

Next up: Basis and Change of Basis, where we ask which set of directions we should use to describe a vector in the first place, and how the same vector looks from a different point of view.

Front: What is the span (column space) of a matrix? Back: The set of all outputs Ax as x ranges over every possible input.

Front: How is rank defined? Back: The number of linearly independent columns, equal to the dimension of the column space.

Front: What is the largest possible rank of an m by n matrix? Back: min(m, n).

Front: What does rank 1 mean for a 2x2 matrix, geometrically? Back: The plane collapses onto a single line.

Front: What does n minus rank measure? Back: The dimension of the null space, the inputs that map to zero.

Front: Determinant of [[a, b], [c, d]]? Back: ad - bc.

Front: What does the absolute value of the determinant measure? Back: How much the transformation scales area (2D) or volume (3D).

Front: What does a determinant of zero tell you? Back: Space collapsed to a lower dimension: the matrix is singular and has no inverse.

Front: What does the sign of a determinant indicate? Back: Whether orientation flipped, as in a reflection.

Front: When does a matrix inverse exist? Back: Only for a square matrix with nonzero determinant, that is, full rank.

Front: What does the inverse do geometrically? Back: It runs the original transformation backwards, so that A inverse A equals I.

Front: Which NumPy function returns the rank of a matrix? Back: np.linalg.matrix_rank.

Front: Why prefer np.linalg.solve over inv then multiply? Back: It is faster and numerically more stable, and it raises an error for singular systems.

Front: What error does np.linalg.inv raise on a singular matrix? Back: LinAlgError.