🔍 A First Look at SVD

By Better Wise

Where We Are: From Eigenvectors to Any Matrix

Last lesson we hunted for vectors where $A v = \lambda v$: directions the matrix only stretches. That was powerful, but it came with two limits. The matrix had to be square, and a real eigenbasis sometimes does not exist. SVD drops both limits. It works for any matrix, square or rectangular, and it always exists.

By the end of this lesson you can:

  • run numpy.linalg.svd and state the shape of each output
  • read the SVD geometrically as rotate, stretch, rotate
  • rebuild a matrix from U, S and Vt
  • drop small singular values for a low rank approximation

Rotate, Stretch, Rotate: The SVD Picture

Every matrix $A$ of shape $m \times n$ can be written as $A = U \Sigma V^T$. Drawing it as motion, applying $A$ to a vector is three moves:

  1. $V^T$ rotates the input into a special orthonormal basis, the right singular vectors.
  2. $\Sigma$ stretches each of those basis directions by a factor $s_i$ and does nothing else. It is diagonal, and it can also squash a direction to zero.
  3. $U$ rotates the stretched result into the output space along the left singular vectors.
flowchart TD
    X["🧭<br/>Input vector x"] --> VT["🔄<br/>V transpose<br/>rotate into right basis"]
    VT --> S["📏<br/>Sigma<br/>stretch by s_i"]
    S --> U["🔄<br/>U<br/>rotate into output space"]
    U --> Y["🎯<br/>Output A x"]

In this lesson

  • Singular Values Rank the Directions
  • Low Rank Approximation in Action
  • What to Carry Forward

About this course

Part of Mathematics for Machine Learning 1: Linear Algebra Foundations — Build the linear algebra intuition behind machine learning: see vectors and matrices as transformations, then compute eigenvalues, projections and SVD yourself in NumPy.

Level
Beginner
Goal
Fundamentals
Language
English
Focus
vectors and norms matrix operations and transformations eigenvalues and SVD geometric intuition NumPy practice
Prior knowledge
High-school algebra (equations, functions), Basic Python or willingness to run short NumPy snippets
Practice
Guided exercises
Time
~4.8h
Voice
a patient university lecturer who draws every operation as a geometric transformation before touching the algebra
Not covered
Calculus and derivatives, Probability and statistics, Deep learning models

This lesson is premium content. Sign in and purchase the course to view the full content. Sign in