# 🔍 A First Look at SVD

By Better Wise

> This lesson is premium content. Sign in and purchase the course to view the full content.

## Where We Are: From Eigenvectors to Any Matrix

Last lesson we hunted for vectors where $A v = \lambda v$: directions the matrix only stretches. That was powerful, but it came with two limits. The matrix had to be square, and a real eigenbasis sometimes does not exist. SVD drops both limits. It works for any matrix, square or rectangular, and it always exists.

By the end of this lesson you can:

- run `numpy.linalg.svd` and state the shape of each output
- read the SVD geometrically as rotate, stretch, rotate
- rebuild a matrix from `U`, `S` and `Vt`
- drop small singular values for a low rank approximation

## Rotate, Stretch, Rotate: The SVD Picture

Every matrix $A$ of shape $m \times n$ can be written as $A = U \Sigma V^T$. Drawing it as motion, applying $A$ to a vector is three moves:

1. $V^T$ rotates the input into a special orthonormal basis, the right singular vectors.
2. $\Sigma$ stretches each of those basis directions by a factor $s_i$ and does nothing else. It is diagonal, and it can also squash a direction to zero.
3. $U$ rotates the stretched result into the output space along the left singular vectors.

```mermaid
flowchart TD
    X["🧭<br/>Input vector x"] --> VT["🔄<br/>V transpose<br/>rotate into right basis"]
    VT --> S["📏<br/>Sigma<br/>stretch by s_i"]
    S --> U["🔄<br/>U<br/>rotate into output space"]
    U --> Y["🎯<br/>Output A x"]
```

## In this lesson

- Singular Values Rank the Directions
- Low Rank Approximation in Action
- What to Carry Forward


---

## Contents

Part of [🧮 Mathematics for Machine Learning 1: Linear Algebra Foundations](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations.md)

- **📐 Vectors, Norms and Similarity**
  - [➡️ Vectors as Arrows, Points, and NumPy Arrays](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/vectors-as-arrows-points-and-numpy-arrays.md)
  - [📏 Norms and Distances: Measuring Vector Size](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/norms-and-distances-measuring-vector-size.md) (sign-in required)
  - [🎯 Dot Product and Cosine Similarity](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/dot-product-and-cosine-similarity.md)
  - [✅ Checkpoint: Vectors and Similarity](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-vectors-and-similarity.md)
- **🔄 Matrices and Transformations**
  - [🧱 Matrices as Transformations and Matrix Multiplication as Composition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/matrices-as-transformations-and-matrix-composition.md)
  - [🧭 Inverse, Rank, and Span](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/inverse-rank-and-span.md)
  - [🔀 Basis and Change of Basis](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/basis-and-change-of-basis.md)
  - [✅ Checkpoint: Matrices and Transformations](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-matrices-and-transformations.md)
- **🔆 Projections, Eigenvectors and SVD**
  - [📉 Projections and Least Squares Intuition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/projections-and-least-squares-intuition.md)
  - [🌀 Eigenvectors and Eigen-Decomposition](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/eigenvectors-and-eigen-decomposition.md)
  - [🔍 A First Look at SVD](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/first-look-at-svd.md) (premium content) — this lesson
  - [✅ Checkpoint: Projections, Eigen and SVD](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/checkpoint-projections-eigen-and-svd.md) (premium content)
- [Course cheatsheet](https://everwise.study/courses/mathematics-for-machine-learning-1-linear-algebra-foundations/cheatsheet.md) (premium content)